Kerby the Herding Dog: In Which We Start (Errorless) Training
By Stephanie Keesey-Phelan, Ph.D., BCBA
In the last installment of Kerby the Herding Dog, I wrote about bringing Kerby to a friend’s farm to see how he felt about sheep. Discovering that he is, in fact, very enthusiastic about sheep and farms in general, it was time to start training. But what to train and how to train it? I’ve already written quite a bit about my desire for a positive reinforcement-based approach, but positive reinforcement training plans can look a lot of different ways.
Trial-and-error learning
I suspect that most of us have had some experience with trial-and-error learning, a situation characterized by the opportunity to make either “correct” or “incorrect” choices in a learning context. The idea with this type of experience is that the learner makes the correct response because it has been followed by something “good”, a reinforcer; and does not make an incorrect response because doing so has resulted in no reinforcement (extinction) or something “bad” (punishment). In the case of herding, a trial-and-error approach might look like putting the dog in with sheep and, either through the sheep’s movement or the delivery of reinforcers from handler, the dog will contact reinforcement for “correct” herding behavior, and the handler will punish “incorrect” herding behavior.
When I reached out to sheepherding instructors in my area, they told me that this kind of approach was not only desirable but necessary. However, as you’ve heard us say plenty of times before, with procedures like extinction and punishment we know there are a slew of negative, damaging side effects we can expect (see references below).
What I wanted for myself and Kerby in our herding training was, first and foremost, joy! I wanted training to be fun. Errors that lead to extinction or punishment are not part of a joyful training experience. To that end, my goal in herding training, and in all training really, is to take an errorless approach.
Errorless teaching
Errorless teaching was first described in an article by Terrace (1963a). Terrace aimed to train pigeons to peck a red light (the correct response), and not to peck a green light (the incorrect response). Instead of a trial-and-error approach in which the researcher presents both lights at the same time and provides reinforcement for pecking the red light, and extinction or punishment for pecking the green light, Terrace first presented the red light only. When the pigeons were consistently pecking the red light, the green light was very gradually faded in. The green light didn’t start at its full intensity, but instead was presented briefly and dimly. Over time, the green light was presented for longer periods at brighter intensities until the pigeon was presented with both the red and green light, and the pigeon continued to peck the red light. In this case, the pigeon didn’t have to learn not to peck the green light by pecking it and contacting extinction or punishment. Instead, the pigeon learned the task (peck red light not green) through this gradual introduction of the green light. This important study demonstrated this method resulted in greater accuracy of responding compared to a trial-and-error method, and that because of a lack of errors, the incorrect stimulus (green light) did not acquire aversive properties.
That is to say, pigeons who learned the discrimination through this method got it “right” more often than ones who were exposed to trial-and-error learning, and they engaged in fewer behaviors suggesting stress and frustration compared to the trial-and-error birds.
**Note: For all of us used to red being the “stop” signal and green being the “go” signal, I kept getting tripped up reading this study where red was the correct choice and green the incorrect! Come on Terrace!
Researchers have gone on to explore the many applications of the errorless approach (Mueller, Palkovic, & Maynard, 2007). In the case of training my little herding dog, what I hoped to do for each skill was to arrange the teaching conditions such that Kerby would be very likely to make the “right” choice - the target behavior - and be very unlikely to make a different choice - an error.
But errors will happen, that’s just life isn’t it?
I should note here that errorless learning is something to aspire to. None of us are perfect. I make mistakes all the time, they don’t call me Stephanie Raining-Treats-From-The-Sky Keesey-Phelan for no reason (this came after a few too many times of bending over to feed a dog and spilling the contents of my treat pouch everywhere. I became very popular with the shelter dogs!). If I could just get a treat from my hand to my dog’s mouth in one fluid motion it would be a miracle, though I’d like to point out that I do not consider this a personal fault, just something I have to work really hard on! Anyway, I digress. Mistakes and errors will happen, but that doesn’t mean we should just throw in the towel and not strive to improve our training methods.
I keep two quotes close at hand for times when errors occur. The first is from Skinner, “The rat is always right.” In this case, we may translate it to: “The dog is always right.” This boils down to mean that any errors that occur are really mine as the trainer. I am the one orchestrating the training environment and the dog, Kerby, is always responding as a result of 1) his learning history, 2) the current context, and 3) his behavioral genetics as a mini Aussie. The second quote, though I’m not sure who to attribute it to, is that “errors are information.” So when errors pop up, my plan is to use that information to consider how I can make changes in the next repetition or session to ensure Kerby’s success and as a result (hopefully!) joy in training with me.
But you can’t control everything!
Not all training with positive reinforcement is designed to be errorless, but that is the goalpost I set for myself. One common misconception when it comes to both positive reinforcement and errorless learning is described by a sheepdog trainer I reached out to:
“Corrections, appropriate ones, are expected to teach the dog what is not acceptable vs letting them try a bunch of stuff until they get a reward marker.”
But if we think back to the pigeons and their red and green lights (Terrace, 1963a) you can see that errorless teaching is not about letting the dog try a bunch of stuff until they get a reward marker. The statement above reflects a misunderstanding about how positive reinforcement works. R+ training generally and errorless learning more specifically is about teaching that is carefully designed and crafted to ensure that the “correct” response is the most likely one.
Going “Errorless”: An Example
So if we’re going to go for errorless, what does that mean practically speaking? There are plenty of ways to go about setting up an errorless learning session but I thought it might be helpful to talk through an example that demonstrates what the process has looked like for Kerby and me. But before I do that, I want to take a moment to provide a few important clarifications:
Although I strive for Kerby to have an errorless experience as a learner, my design of the training session is far from “errorless.” As you’ll see below, I start with a behavioral definition and then try to figure out if there are any current situations in which the behavior is already happening that I could pull into my session. But as noted above, I’m just a human and I don’t always observe every relevant variable. Sometimes I miss something and get an “oops” moment because Kerby does something I didn’t expect or wasn’t planning for. In those cases, I typically shake my head at myself, laugh, and take a break to figure out what a better training set up might be. In that regard, though I’m aiming for Kerby to get it right with each repetition of a skill, my process of designing a training session remains very much trial-and-error.
The next thing is, I find it enormously valuable to tackle this sort of thing as a group project. In this case my colleague and friend Maia Huff-Owen, fellow behavior nerd and herding trainer/ enthusiast, graciously worked with me to develop these training plans. The benefits of another set of eyes are innumerable. Maia and I worked together via Zoom and she observed things I missed, asked insightful questions, and brainstormed with me until we found a procedure that resulted in an errorless approach (and a pretty darn good version of the behavior, if I do say so myself!). Locally, I was fortunate to work with another wonderful colleague and friend, Kim Palermo who is training her lovely collie, Jane, to herd. Kim and I set up training dates together with our dogs to practice herding skills at Kim’s training facility and at her home with her flock of ducks. This in-the-moment training, feedback, and brainstorming was invaluable. If you have the opportunity, I can’t recommend working with a trusted friend and trainer enough.
Okay enough with the caveats, let’s look at the creation of a real herding training plan!
Going “Errorless”: Nuts and Bolts
The first thing we have to start with is a behavior that we want to teach. Kerby already has a strong foundation of Control Unleashed and obedience-related skills and so I was ready to move on to herding-specific skills. The first one I decided to teach was “walk in.” We aren’t going to be very successful in training if we don’t know what our target behavior looks like. To that end, we’ll start with a definition.
Definition of Walk-In:
Walk in is one of the foundational herding skills. It involves a dog slowly walking directly toward the stock. The dog should not run, lunge, chase, or try to bite the stock. The dog may move in a crouched position that looks like a predator stalking its prey. When this behavior is happening successfully, the stock should move away from the dog calmly (i.e., not exhibiting stress signals that we might see with that type of stock) and at a comfortable pace (i.e., a walk or trot).
Below is a video example - watch from 0 to 25 seconds!
Keep in mind that the dog in this example is a border collie and Kerby is a miniature Australian shepherd. Why does this matter? Because these different breeds move differently, so the same behavior is going to look physically different for Kerby, but the goal is the same: Kerby moves toward the stock at a walking pace.
Training set up:
For this particular behavior, the biggest barrier was that I had spent SO MUCH TIME reinforcing Kerby’s orientation and eye contact toward me. While this is a great skill for most dogs, it is problematic with herding when the dog is supposed to be watching the stock. I couldn’t quite figure out how to get Kerby to move away from me toward a target without pausing and looking back at me to access a reinforcer.
So the first question was: where is the behavior already happening? For Kerby, I was hard-pressed to think about times when he both looked and moved away from me - except maybe when chasing animals in the woods. A logical progression from this point would be: why not go out and teach this behavior directly with sheep? Sheep should capture his attention, and since he seemed to have some instinct for herding (see previous post), wouldn’t that be the best starting place? I decided almost immediately: no, I wouldn’t train the walk in with actual livestock. Here are three reasons why:
Although sheep might elicit some behaviors that would be desirable, potentially including a walk-in, I haven’t taught Kerby how to follow my cues to move in certain ways around stock.
In this scenario, the sheep would likely elicit a variety of behaviors that I am not looking for and as a result, Kerby would be likely to make lots of mistakes. If Kerby makes mistakes, one of two things can happen:
If I do nothing, the sheep’s response to Kerby may reinforce the “mistake,” resulting in it happening more often in the future - an undesirable outcome.
If I intervene using punishment, the learning environment is being conditioned as aversive, and I am likely to see a number of negative side effects associated with the use of punishment.
I know it is possible to teach this skill away from sheep conceptually because we have so much research demonstrating similar types of generalization of skills across contexts and settings. AND, I have met brilliant people who are taking this approach with sheepherding specifically.
So if I wouldn’t be using sheep to capture Kerby’s walk-in behavior, what would I do instead? I brainstormed with Maia and we considered two major components of this behavior:
Kerby needs to move at a walk or trot pace
Kerby needs to move in a straight line
That straight line is toward a visual target (eventually sheep) and, at least at the beginning, away from me
What we decided to do was borrow and modify a Control Unleashed activity called Super Bowls. In the Super Bowls set up, the dog moves forward from one visual target to another, often in a straight line, and at a walking pace. The only difference between Super Bowls and what I wanted to do was that I wanted Kerby to move along looking forward instead of stopping at each target and looking up at me like he does in a Super Bowls set up. So we did the following:
We changed the visual targets from the silicone exercise dots that I used for Super Bowls to small traffic cones evenly spaced out across approximately 10 feet.
Instead of pausing at each visual target and waiting for eye contact as I do with Super Bowls, I would move with Kerby to each visual target and drop several treats
As Kerby ate the treats at the target, I would jog ahead to the next one and start dropping treats there.
I would repeat this pattern until I got to the end of my line of cones.
Because I would be moving slightly ahead of Kerby, I videotaped all of my sessions so I could go back and see where he was looking as he moved from one cone to the next. My goal was for him to look straight forward to the cone where treats would appear instead of up toward my face or my hands.
When Kerby was fluently engaging in the target behavior, I would begin to fade out the number of cones until I just had one cone 10 feet away, pre-loaded with treats and I would send Kerby from my starting location to the cone.
How did it go?
You can see the process in the video below.
Here are some big takeaways:
Generalization for better or worse: Last year I taught Kerby to knock over vertical objects. I did not use traffic cones, but when I placed the cones on the ground for training this was one of the first things that Kerby did *face palm*! Ultimately I was still able to use the cones, it just took some extra practice and quick delivery of reinforcement for the behavior I wanted (walking from cone to cone).
What about when errors happened? It depended on the type of error:
Kerby hesitates: This came up when I faded my movement from the training context. I waited 2-3 seconds to see if he would keep going. In all cases he did keep going and hesitation didn’t persist within the training session
Not waiting for the cue: In most cases I had Kerby sit next to me to start, and prior to starting each trial the cones ahead had been pre-loaded with treats. Sometimes Kerby bounded forward before I gave the “walk in” cue. When this happened, initially I called Kerby back to me and reset. But I found that when I did this it was almost always followed by a trial where he hesitated to move forward which I did not want. What I decided to do was just let him go when this happened, but to provide more reinforcement for sitting next to me in the future. After a few sessions Kerby remained at my side much more reliably.
In the grand scheme of things, this process felt really good for me and, with the exception of times when I called Kerby out of the “walk in” because he “broke the sit” I rarely saw stress responses in training. Kerby tended to move with confidence at a walk/trot pace.
What’s next?
The example above shows just one of several herding skills that I have been working on using an errorless approach with Kerby. For the walk-in behavior in particular, we have worked on transitioning the reinforcer to a tug toy and at the time of this writing (April 2024) we are almost ready to try this out with livestock!
Although the training process continues on, the next post in this series will be about Kerby’s first clinic with Barb Buchmayer at the Canine Center at Hessian Hill in New York last fall. Kerby and I are on our way!
Missed prior posts? You can click on the links below to go back and read about the steps that led to this point in our journey.
References: Errorless learning
Mueller, M. M., Palkovic, C. M., & Maynard, C. S. (2007). Errorless learning: Review and practical application for teaching children with pervasive developmental disorders. Psychology in the Schools, 44(7), 691-700.
Terrace, H.S. (1963a). Discrimination learning with and without “errors”. Journal of the Experimental Analysis of Behavior, 6, 1–27.
Terrace, H.S. (1963b). Errorless transfer of a discrimination across two continua. Journal of the Experimental Analysis of Behavior, 6, 223–232
References: Side effects of errors
Azrin, N.H., Hutchinson, R.R., & Hake, D.F. (1966). Extinction-induced aggression. Journal of the Experimental Analysis of Behavior, 9, 191–204.
Ducharme, J.M. (2003). “Errorless” rehabilitation: Strategies of proactive intervention for individuals with acquired brain injury and their children. Journal of Head Trauma Rehabilitation, 18, 88–104.
Kelly, J.F., & Hake, D.F. (1970). An extinction-induced increase in an aggressive response with humans. Journal of the Experimental Analysis of Behavior, 14, 153–164.
Lerman, D.C., Iwata, B.A., & Wallace, M.D. (1999). Side effects of extinction: Prevalence of bursting and aggression during the treatment of self-injurious behavior. Journal of Applied Behavior Analysis, 32, 1–8.
Rilling, M., & Caplan, H.J. (1973). Extinction-induced aggression during errorless discrimination learning. Journal of the Experimental Analysis of Behavior, 20, 85–92.
Weeks, M., & Gaylord-Ross, R. (1981). Task difficulty and aberrant behavior in severely handicapped students. Journal of Applied Behavior Analysis, 14, 449– 463.