When Can I Stop Using Treats?

“When can I stop using treats?” I’ve seen this question asked and answered in a couple different places recently, and thought I might add another layer to the discussion. The glib answer is “never,” and I often also see “would you keep working if you didn’t get paid?” Some more nuanced answers convey the idea that you will need to keep using treats when you are asking the dog to do a behavior that isn’t “natural” or that the dog does not “prefer.”

I want to suggest that an even more useful approach might be to think about where the reinforcers are coming from. (When I say reinforcers, I mean consequences that increase or maintain the frequency of a behavior, so if what you think should be a reinforcer isn't doing the job, it's not a reinforcer.) We can only infer what dogs “prefer” from what they do, and they do what they do (largely) because of the outcomes it produces in their environment. We do know that behavior that is not reinforced is likely to weaken; that is the adaptive nature of behavior.

Very often, when we are teaching a new skill, we use treats because we are breaking a bigger, more complex behavior down into small steps that don't produce any other clear benefit to the dog. As the skill is built, however, the dog can reach a point where they are able to do enough behavior to directly preoduce reinforcement.

So, my answer is: You can stop using “treats,” if that’s what you used to get the behavior started, when the behavior is big and strong enough to carry the dog to reinforcers that are more likely to occur as a typical product of the behavior.

Here's an example: with Rody and Zuzu, who jumped, barked, and spun while their meals were being prepared, I broke down staying on the floor into small steps, reinforcing with bits of their dinner or other treats, until they could stay put long enough for me to prepare dinner and serve it to them. (Note: the videos are labeled first, second, third session etc. but there might've been a couple others. I basically just trained with almost every meal for a few days.)

That's kind of an easy one, because the final reinforcer was still food. But what about behaviors that ultimately might not be done for food if you weren't "training"? For instance, many dogs will initially try to escape or investigate any strange thing you touch them with, and in order to be able to even start to bring a brush, a stethoscope, or a needle toward them, you may need treats to reinforce small amounts of holding still.

But when you teach a dog to hold still long enough for a couple of brush strokes, they may learn that the feel of the brush going down their back is kind of nice. That is the naturally occurring reinforcer for holding still for brushing. We might say that dog “likes” brushing, but it is more helpful to recognize when holding still for brushing is reinforced by the feel of the brush going through the hair or down the back.

This is less likely to happen with a needle poke, so you probably will still want to use treats for the same behavior of holding still when the outcome is a sharp jab in the butt.

Another example: You teach a dog to walk on a loose leash using treats. When the dog can walk on a loose leash long enough to get from a bush to a garbage can to a fire hydrant, access to the things he wants to sniff can likely take over as reinforcers.

TLDR: When the behavior will not bring the dog into contact with other reinforcers, keep using treats.

Thanks for Barking 2.0

A number of years ago, I wrote a short blog post for clickertraining.com called How to Get Your Dog to Stop Barking—by Thanking Him for Barking.

Over the years, this post has grown legs. The procedure it describes is often referred to as the "thank you protocol," and sometimes to my embarrassment "Kiki Yablon's thank you protocol," though others have done similar things. (One example that comes to mind for me is Irith Bloom's video called "Click for Barking.")

The original post was very abbreviated; it was written with a word limit, as a quick hit for National Train Your Dog Month. I didn't have room for anything about the reasons I thought it worked, or to go into detail about what I think is actually the most interesting piece, which I sort of tacked on at the end as "extra credit." I've been meaning to expand on both of those things for a while, and I guess today's the day.

The idea occurred to me, not surprisingly, when I was having a problem with my own dog, Pigeon. Our whole back yard is fenced with 6-foot solid wood, but down the gangway is a chain link gate with a view to the street and sidewalk. When Pigeon saw people or other dogs in that narrow window, she would run to the gate and hurl herself at it, barking and snarling. The likely reinforcer for this behavior seemed to be that the people and dogs went away.

Pigeon did similar behavior on walks at the time, but there I was having some success. I was able to get ahead of the behavior--to preempt rather than redirect, and reinforce just looking at the people or other dogs, or at me, or sniffing. But in the yard, unless I stood out in front of the house, I couldn’t tell anyone was coming until Pigeon was already sprinting to the gate. And I also couldn’t control whether the passers-by continued on their merry way, thereby strengthening her behavior. This is often also the case with dogs who bark out of windows or at sounds in the hallway of an apartment building.

I felt like I needed to interrupt this behavior, because it was scary for some of the passers-by and, frankly, embarrassing for me. But my recall wasn’t actually working to call her out of it, and I was a little worried that if it did actually work, I might teach her to bark to get me to call her for treats. Now I would say without hesitation: Great! If I can in fact get her to bark for food instead of to make stuff go away, I’ll be in a good position to use food to reinforce something else.

I gave it a shot anyway. And, fair warning, I was sloppy (call it experimental if you want to be generous), as I often am with my own dog. Rather than plan sessions or brush up on the distance or distraction level on her recall first, I just committed to being outside if Pigeon was outside, with high value food on me, over the course of a summer. My husband also wove some vinyl straps through the chain link, which made it harder to see some things, but couldn’t completely block Pigeon’s view. As soon as Pigeon started to run to the gate, I ran after her and gave my recall cue (a whistle) from literally right behind her. If I got so much as a tense sideways glance, I fed handfuls of Stella and Chewy’s freeze dried patties or boiled chicken.

Pretty soon it became easier for her to look when I whistled, and I was able to fade myself gradually back to the picnic table. So now she was still running toward the gate, and sometimes getting in a few barks--but I could interrupt pretty early in the sequence without getting up off my butt. That alone would have been a vast improvement in our time together in the yard. But it looked to me like she was in fact barking at passers-by to get called back for food, and so I decided to test it: I stopped calling her right away when she barked to see what she would do next. She ran as far as she usually got when I called her, about halfway down the gangway, and when I didn’t whistle, she turned and looked at me. I immediately called her then.

When I saw that, it brought to mind to a grainy old video that I had seen while enrolled in the Karen Pryor Academy, where I started my formal dog training education in 2011 and where I am now faculty. Teaching in a hotel conference room, Pryor used audience members to demonstrate both how cues could serve as conditioned reinforcers to hold together a sequence of behaviors and what happened when one of the cues disappeared. The gist was that when a cue failed to come at the right time, the sequence folded in on itself in a predictable way—the behavior before the missing cue got shorter, and the learner moved along more quickly from it to the next behavior, which was closer to the click and Hershey’s kiss she got at the end of the sequence.

This makes sense to me, because we know that behaviors that are no longer reinforced will weaken ... eventually. We also know that when you are going to stop providing reinforcement for one behavior, you will minimize the problems that vacuum can cause (google “extinction burst”) if you can provide a clear and easy alternative path to reinforcement. If the elevator you normally take upstairs is not coming when you press the button, before your button pressing weakens, you may press more or harder, curse, kick the wall, or call the manager. But if there’s a sign posted saying another elevator down the hall is working, then you may simply walk down the hall and press that button.

When behaviors frequently are cued in a certain order, and one of the cues is too late to reinforce the behavior that came before it, we often see our learners move along to the next behavior down the line. And if that behavior produces the reinforcer, we will often see the learner do that behavior more in place of the one that didn’t work. For instance: people taking beginner dog classes often learn to teach down once their dog has learned to sit. They often practice by cueing sit, then cuing down, then giving their dog a treat. When they are late with that down cue, you will often see the dog go ahead and do the down. People may then give the treat anyway, because the dog did something it was trained to do, even if it wasn’t what was requested. And now they have a dog who lies down when they say “sit.”

While this sort of outcome is bad news in, say, an agility performance, in my case I wanted Pigeon to run and bark less or less intensely, and come back to me faster, when someone passed by the gate (the equivalent of the "sit" cue in the example above). So when I didn’t call her after she ran (the equivalent of being late with the down cue), and Pigeon turned back (the equivalent of lying down anyway), I reinforced deliberately and with gusto. She would take a few steps or look toward the gate, and often didn’t bother with any barking. And she began to look at me more quickly and sometimes start to come my way, and when that happened I moved the reinforcer down the line a little more. You could think of this as a shaping procedure--once you control the reinforcer, you can then reinforce successive approximations starting with behavior that is less intense barking and ending with turning and coming to you.

I think it's possible that in some cases there is some transfer of stimulus control happening--a cue transfer, in training terms. Many of us have used the "new cue, old cue" procedure to add a new signal to a behavior that already has one. If your dog starts coming when they see the trigger, the passer-by, which has routinely and closely preceded the recall cue, could be conceived of as the "new" cue for recall.

Another factor when this procedure is successful may be that passers-by may have come to predict the treats. So the passers-by may have started to elicit some of the same involuntary physiological responses that treats did (though most of those would have been invisible to me), and the fact that the they predicted treats may have made it less reinforcing to Pigeon for them to go away.

Of course all of this may be wrong, or some mechanisms may be in play in some cases where this procedure works, and others in other cases. I'm spitballing here based on my understanding of principles, but haven't tried to rule out one mechanism or the other in any given case.

When I wrote the “quick tip” for clickertraining.com, I glossed over the last part of the process, which actually I find the most interesting, because of the amount of explanation I thought it required. But because many people seem to be trying this on their own now, with only that brief post or secondhand descriptions of it to guide them, I've been feeling like I need to elaborate.

So here are the instructions I now give. They can and should be adjusted for individual situations. And you should have a good observation-informed guess about the purpose and context of your dog's barking--this isn't something I just apply for barking "in general." (For more on how you might decide what to do about barking based on when it happens and its function, see The ABCs of Barking.)

I know that this post so far may feel a bit like reading through three generations of someone’s family history just to get their damn pot roast recipe. But as with cooking, if things aren’t going as expected, it suddenly becomes important to understand not just what the recipe says to do but why. (Sarah Stremming, Marissa Martino, and Lisa Mullinax recently did a very entertaining and informative extended riff on this metaphor.)

Step 1: Preventive Management: Temporarily prevent rehearsal of the behavior as much as possible when you are not actively training or ready to train. For example:

  • For visually stimulated dogs, this may be as simple as closing the blinds. Translucent window film halfway up the window can still let light in and preserve the humans’ view.
  • For dogs who respond to sounds, close windows. Play music or white noise, or run a box fan or window AC.
  • Block access to any window or door where your dog tends to bark using doors or baby gates.

Sometimes, this is enough. If this solves your problem, it's perfectly fine to stop here.

Step 2: Preparation

  • Choose a cue that does not already have a long history of not working to get your dog’s attention or get them to come. “Thank you” may be a good choice because it’s unlikely you’ve said it to your dog, but there’s no magic imbued in those particular words.
  • Prepare some soft, small, highly preferred treats and store them in a location that will become predictable to your dog. This could be a pocket or treat pouch, in which case you are the location. Or it can be a container on a mantle, shelf, or console table. It can also help to have a target like dog bed or mat directly under the container and always deliver the treats there—the predictability makes it easier to know what to do.
  • I like these OXO pop-top containers for this because they are easy to open quickly, and as a bonus, they click!

Step 3: Training Prerequisite Behavior Start when your dog is not already barking, is somewhat attentive to you, and is in the mood for food.

  • Stand or sit near your dog.
  • Give the cue, then dispense 10-15 treats--or go directly to the designated location and dispense them there. Make sure the cue precedes any movement to dispense the treats. It needs to have predictive value.
  • Give the treats one after the other right in front of you if you are the “location,” or scatter them on the mat or bed.
  • Dispense the treats whether your dog comes to you or follows you to the location or not. This is very important, as this is how your dog will learn to go to the location.
  • When the dog starts to look excited upon hearing the cue, you can start to say it when the dog is not already paying attention to you.
  • Continue to deliver the treats reliably if you said the cue, whether your dog has looked or come to you.
  • Practice from different parts of the house. Practice near the window or door where the dog usually barks. Practice with no one outside. Practice several times a day when your dog is not already barking. Practice with distractions that are interesting to your dog but not the things they usually bark at.

Step 4: Reinforcing the Barking (Or, Even Better, Precursors to Barking) When your dog whips around quickly upon hearing the cue and starts to move toward you or the location in anticipation of the treats, begin to work at times when there are things outside that your dog would normally bark at.

  • Give the cue the instant your dog begins to bark, or even better, right when he alerts or starts to move. So: before he starts barking or after just a few barks.
  • If he does not respond, get closer to him before cuing again (or next time), use higher value treats (next time), and practice more near the problem location without the trigger, or with less intense versions of the trigger (e.g., passers-by across the street, known people instead of strangers)
  • When he's responding to the cue with the trigger present, start to increase your distance from the dog again a few steps at a time. Work farther away only when your dog is doing well with you at the current distance.
  • The ideal time to give this cue, once trained, is as soon you see the dog alerting to a sound or starting to go to the window. But because this procedure is best for situations where you can’t get ahead of what your dog is perceiving, it’s likely you’ll be giving it after a bark or three. That’s OK! An initial goal is for the dog to come away from the window to find you on cue after a bark or two at most.
  • Make sure that your dog is perceiving an actual stimulus outside. Don’t thank your dog for barking when you didn’t hear and can’t see anything, or you may create a different barking problem.

Step 5: Moving the Reinforcer Down the Line When your dog is responding rapidly when you cue, or you see that he starts to look at you as if expecting to be thanked after a few barks, test.

  • Don’t call your dog when he first barks.
  • Watch for the dog to turn back to you, or turn away from the window, after the usual amount of barking that he does before you cue him. He is likely to do this after the usual number of barks that you reinforce by calling him.
  • When you see that behavior, immediately cue him! Mark for any movement and reinforce in front of you or at your treat container location.

You may see the barking grow more perfunctory, or that your dog even begins to skip it and just look toward you or move toward you or the treat container. Notice and capitalize on this! Reinforce even tiny movements in the right direction immediately.

For additional thoughts since this post was first published, see Thanks for Barking: Addenda.

Do You Turn Your Back on Your Dog When He Jumps? Is He Still Jumping?

Screen Shot 2020-07-14 at 11.18.05 AM.png

When your dog jumps, common advice--which just came across my feed again this morning--is to turn your back, and then when the dog stops jumping, ask for a sit.

But any strategy that relies on waiting for the unwanted behavior to occur first is imperfect, as discussed in this blog post. Let's look at how this applies in the case of jumping:

Ignoring an attention-seeking behavior (which not all jumping is) that's been working doesn't automatically produce a preferred behavior; in fact, the animal is likely to try harder, or try other behaviors that serve the same purpose. With jumping, many households cannot consistently ignore that more intense or varied behavior, and will end up reinforcing it when they fail to ignore.

If you're going to withhold reinforcement for an unwanted behavior, it'll work best if you have already taught your preferred replacement behavior AND made clear that it can produce the same outcome or something even better.

Also, let's look at the commonly advised sequence more closely, and think about what is actually likely to be learned even if it is properly executed.

Cue: Person comes in the door Behavior: Dog jumps Outcome/cue: Person turns back Behavior: Dog puts feet back on floor Outcome/cue: Person turns around and gives sit cue Behavior: Dog sits Outcome: Person pets dog

Quiz (answers at the end of the post):

  • What is the cue for putting feet on the floor? (1)
  • What behavior does the dog have to do in order to receive that cue? (2)
  • Cues are opportunities for reinforcement, and can thus serve as reinforcers themselves. What behavior will turning one's back reinforce? (3)

What most people want is for the person coming in the door to be the cue for the dog to keep its feet on the floor. This can be achieved efficiently by pre-teaching the dog that four on the floor (or jumping up onto an ottoman, or grabbing a toy, or sitting, though that may be much harder for a dog who has probably been alone and inactive for hours) is a good way to get attention, and then giving attention for four feet on the floor right after you come in the door.

If necessary ,you can use a gate or ex pen around the door to prevent jumping temporarily while you teach the new contingency or increase the duration of feet on the floor.

Quiz answers:

  1. the cue to put feet on the floor is the person turning her back
  2. the dog has to jump to get that cue
  3. jumping is likely to be maintained

Knowledge, Experience, and Reinforcement

Hierarchy of Behavior Change Procedures

Hierarchy of Behavior Change Procedures

There are currently two discussions happening in the animal training world that are related to how people should choose what approaches to try first when they decide an animal’s behavior needs to change. One of the discussions involves a magazine columnist—at the magazine where I held my first job, though our tenures did not overlap—who wrote a personal essay about his decision to use a shock collar on his dog after first failing at certain goals with unspecified positive reinforcement approaches. The other discussion is about the Hierarchy of Behavior Change Procedures, a visual tool developed by one of my mentors, Dr. Susan Friedman, to help animal caregivers choose the effective approach that lets the learner retain the most control and also produces the most behavior that we might associate with good welfare and “positive” emotions.

These guidelines and the more nuanced article originally intended to inform their use have been used most recently as the basis for the "LIMA standard," a set of professional ethics adopted by a number of prominent animal training organizations, and as such of course should be revisited and reexamined regularly. For me, one of the most salient points of a new article Dr. Friedman published about the hierarchy this week is that the use of procedures that are ethically preferable, such as reinforcement (procedures that strengthen desired behavior), may require more expertise and experience than the use of punishment (procedures that decrease or suppress behavior), which we all get plenty of practice at from the day we are born, and which, when it is effective, can provide immediate gratification. While learning is a lifelong process for trainers as well as animals, she argues, those who are paid to dispense behavior advice in particular should seek education, experience, and mentoring to make sure animals don’t pay too dearly for whatever our current shortcomings may be.

The idea that how effectively one can use reinforcement depends on the knowledge and skill one has at the time came to mind when reading criticism of the magazine article by professional dog trainers (which is what called my attention to it), and then again when editing the following short before-and-after video (shared with permission from my client). Although the video shows only the beginning of a training process, I think it is a good example of how reinforcement-based strategies may succeed or fail, or fall somewhere in between, based on nuances that rely on education, experience, or both. (I would also add that while most humans are experienced with punishment, using it while minimizing the risk of fallout depends on similar nuances and most of us are not so good at that.)

The dog in the video below generally behaves to escape and avoid having her paws handled and her nails trimmed with any kind of handheld device. Her devoted and patient owner is a previous client, and with very minimal help from me a couple years ago, she taught the dog to file her own nails using a scratch board. But she still needs a way to tackle the dew claws, and so she had recently embarked on a plan to help her dog feel comfortable having her nails Dremeled. She asked for my help because she got stuck. I recognized where she got stuck, because I’ve been stuck there before. In fact, I got stuck in a similar place with my own dog years ago, and, full disclosure, I didn’t work through it in as pretty a way as you’ll see below. But I have more knowledge and experience now than then I did then, and I hope I’ll have more next week, next month, and a year from now.

Here’s the video. The first clip is the beginning of our session, conducted via Zoom on May 28, 2020. The second clip was sent by the owner on May 30:

Some excellent thigs the owner was already doing:

  • Starting with the dog in a settled base position that was already trained and associated with lots of good stuff
  • Working in a space where the dog could leave at any time
  • Breaking her goal down into small steps--she has also already separately associated the sound of the Dremel with bacon and begun using the Dremel to grind penne to introduce the way the sound will change when it is used on a nail (a technique I like to call "pasta alla Torelli" because I learned it from Laura Monaco Torelli)
  • Moving deliberately so she did not surprise the dog
  • Using food items that her dog loves. Bacon!
  • Asking for help when she got stuck

Here are all the little things we changed after reviewing where things stood, in order to get to where things are now:

  • Skip the clicker and just use hand movement to mark behavior, which will improve timing of reinforcement.
  • Elevate the mat on the couch so that the dog’s paws are hanging over edge. That way, she doesn’t have to stand up or lift her tiny leg up awkwardly from a down position on the floor to offer a paw. She can then remain in a comfortable and stable position while her paw is in the hand.
  • Aim for the dog to initiate contact, rather than to “allow touching.” (Note: The dog’s willingness to do this this likely came from my client’s prior simple pairing of touching + treat.)
  • Wait to start the next trial until the dog finishes eating, so that the prior treat is not still in play before the next contact is made. That way the dog doesn't have to choose between eating and leaving and we can more clearly see whether she is participating voluntarily. We are also less likely to make her suspicious of treats because they're not predicting being touched.
  • When she’s done eating, watch what she does. Pick a behavior, or several, that will cue you to present the next trial, such as looking at you or lifting a paw. If you don’t see the behavior you’re looking for, or if it comes hesitantly; adjust the plan.
  • To get rid of the “swiping” behavior, the product of withholding the click to get duration, temporarily reduce the requirement for the paw touch to smallest success point (h/t Laura Monaco Torelli), the smallest perfect behavior (h/t Mary Hunter, though I may be misremembering her language), the shortest antecedent-behavior-consequence loop (h/t Alexandra Kurland), or whatever you want to call it. Reinforce the moment the paw makes contact with the hand, which will actually mean anticipating when it is about to make contact and moving the treat hand a little early.
  • Deliver the treat low, under her nose, so that she does not have to lift her head or stand up to eat, and can easily keep the paw in your hand.
  • Rebuild duration from this point by treating for just a second or two of leaving the paw in your palm.
  • Begin to gradually increase the time between treats.
  • End the trial while the paw is solidly resting in the hand by removing your hand and stopping treats.
  • In the space created when treats can be delivered farther apart, introduce approximations toward holding and manipulating the paw. (More nuances of this step have since been discussed, but are not fully demonstrated in the video.)

Further clarifications I saw the opportunity to make after watching the second video:

  • Whatever you’re going to do, that’s the deal for that trial. If you are ready to increase your criterion, do it on the next trial, and not in the middle of the current one. Don’t change the deal mid-handshake.
  • When you hear yourself thinking “man, she’s doing great, let’s get one more,” toss a treat off the mat and see if she comes back quickly and enthusiastically to ask for more. If she doesn’t, end the session and adjust the plan.

We still have a ways to go before I would call the dog “comfortable” with paw handling, but this is such great progress for a day’s work, and small changes made a large difference. Further, I’m certain that my peers and mentors, including those named above, would see even more opportunity for improvement. (In fact, I can hear Laura Monaco Torelli in my head now, reminding me to intersperse other fun and well-known behaviors.) There’s almost always a way to do better.

Lest this inspire despair, know that in many if not most cases, your ever-changing best will be plenty. This is evident from the number of people currently living more or less happily with dogs and vice-versa, despite a lack of training and less than perfect behavior on both sides and yes, even the occasional aversive stimulus.

But my point here is: If you’ve tried reinforcement and it has failed, there may still be a lot you can do before you start assuming you must use more aversive procedures, and some of it can make a big difference fast. (Notes: 1. This client, as far as I know, was not considering more aversive procedures. 2. I'm not saying the magazine writer did or didn't try such things, or how the use of the shock collar affected his dog's welfare; I have no idea.)

As far as the application of the hierarchy goes, it is worth noting that this process of giving the learner more control involves both positive and negative reinforcement procedures. The dog can take her paw away and that behavior will be allowed to serve its function of escape or avoidance; she won’t need to squirm or bite in order to stop the procedure if we miscalculate what she’s ready for. Or she can meet the current training criterion and get treats, which also happen to be followed by release. And she can leave the session entirely any time she likes.

Ideally criteria are set carefully, so that she doesn’t choose to take her paw away or leave very often, if at all, and we see loose body language and low latency. But I would also be interested, based on some very cool work by both Ken Ramirez (2017, teaching a beluga whale to say “no”) and researchers in the human world (Rajaraman et al., 2019, “enhanced choice model”), in seeing what might happen if we gave her the same treat for taking her paw away or some other alternative behavior as well.

Are there arguments to be made that a process like this is not always the least intrusive, or least restrictive, effective approach? Sure. First, it remains to be seen if this will be effective in the long run. Second, if this were a dog who had come from the shelter with nails curling into her paw pads, it would probably not be the option that immediately provided the best welfare. I would likely recommend having a kind vet or groomer get the job done in some other context, and then start work in a different setting on cooperating with nail trims. But happily this dog has nail-filing in her repertoire, and an owner with the resources and time to improve her own skills.