Procedural skills (i.e., skills that are learned incrementally through trial-and-error) represent a huge and important subset of behavior that underpin many of the most fundamental aspects of how organisms survive and prosper. However, procedural skills are also thought to underly maladaptive states such as addiction. There is overwhelming evidence that once a procedural skill is learned, it is extremely difficult to unlearn. In this dissertation, we propose and test a biologically-detailed theory of how such skills are learned, and why they are so difficult to unlearn. The theory we propose in this dissertation is characterized by a number of key features. First, procedural skills are learned at cortico-striatal synapses. Second, cholinergic interneurons in the striatum known as TANs (i.e. Tonically Active Neurons) tonically inhibit cortical input to striatal output neurons, and thereby gate the learning and expression of striatal dependent behaviors. The TANs are driven by cells in the centremedian and parafascicular (CM-Pf) nucleus of the thalamus, which in turn are broadly tuned to features of the environment. Finally, the model assumes that learning at all cortical-striatal and Pf-TAN synapses is driven by a dopamine signal that is sensitive to action-outcome contingencies. We examine the ability a formal computational model based on these features to account for key results from two well known classes of procedural skills (i.e., appetitive instrumental conditioning and information-integration category-learning).