Showing posts with label reinforcement. Show all posts
Showing posts with label reinforcement. Show all posts

Monday, October 3, 2011

Clicker Training Do's & Don'ts

DO give a treat for every click, even if you clicked by mistake, you can sort out the mistaken click later.  
          DON'T give treats during training unless you clicked first because it 
          weakens the training bargain.

DO understand that click ends the behavior and so it is expected that your dog will stop doing the behavior when you click.   
          DON'T try to shape more than one criteria at one time.  

DO use the clicker to reinforce attention from the dog.  
          DON'T use the clicker as an attention getting device; you could be
          reinforcing behavior you don't want. 

DO teach your dog the meaning of a keep going signal (KGS) and use this in training to tell the dog to continue what they are presently doing and that a click/treat is coming. 
          DON'T use the clicker as a KGS because your dog will be confused.

DO reinforce behavior you want to see more of, rather than ignore the dog when it is behaving well.  
          DON'T punish misbehavior, you could be reinforcing the behavior
          with attention. 

DO manage your environment and training to set your dog up for success and reward.  
          DON'T "proof" your dog by setting him up to make a mistake. 

DO use a high rate of reinforcement and your dog will be eager to train.  
          DON'T worry about spoiling your dog with too much reinforcement. 

DO train off leash whenever possible in a safe environment. 
          DON'T use your leash as a training tool. 

DO have a clicker on your person at all times and treats stashed in many places so you can reinforce good behavior whenever it occurs.   
          DON'T miss impromptu training opportunities.  

DO make a list of all of the things your dog finds reinforcing and rank them in order -- remember this is the list for your dog's preferences, not what you think they should be. 
          DON'T limit the kinds of things you use for reinforcers.  

DO carry a variety of treats so you can use a special treat for a special effort (such as ignoring distraction) or for sudden leaps into brilliant performances. 
          DON'T become predictable to your dog about what you will treat with.  

DO be a benevolent leader who sets fair rules and limitations and teaches the dog what they are and how to live within those rules. 
          DON'T be a malevolent dictator who changes the rules or behaves
          unpredictably or unfairly to the dog.  

DO split goal behaviors into fine criteria and shape from one to the other in a steady progression.  
          DON'T lump behaviors and wait for complete or flawless
          performances before you reinforce them.  

DO plan your training session and how you will shape each behavior so you can reinforce the progression as it occurs. 
          DON'T miss out on those sudden intuitive leaps skipping
          several steps, this is the perfect time for a jackpot.  

DO concentrate on the behavior you want to see your dog do, instead of the things you don't want to see.  
          DON'T seek to get rid of an undesirable behavior without having
          a new behavior to replace it with. 

Do's & Don'ts /with Explanations


DO give a treat for every click, even if you clicked by mistake, you can sort out the mistaken click later.  Clicker training is very forgiving.  It is far better to give an occasional reward that wasn't earned than to dole out punishment that wasn't deserved.

DON'T give treats during training unless you clicked first because it weakens the training bargain.  Your training bargain is: click equals treat.  In the beginning of training, you want to carefully keep that bargain.  Later on you can be more flexible AFTER your dog knows her "job".

DO understand that click ends the behavior and so it is expected that your dog will stop doing the behavior when you click.  You can work on other criteria (such as duration of stays or heeling) at another time by using a "keep going signal" (KGS).  I use the word "yesssss" (with an extended "s" on the end) to tell the dog she is getting closer to the click and treat.

DON'T try to shape more than one criteria at one time.  Dogs learn better, faster and more consistently if they are presented with just one challenge at a time.  So if you've been teaching sit stay and your dog is able to do it for one minute and now you want to take it on the road to teach it in a distracting environment, do expect to lose your duration.  Start with a much shorter duration and build the time up gradually under this new distracting condition.

Gimme here:  My person taught me to stay by first teaching me to wait for permission for important stuff, like breakfast.  So then the day she taught me stay was very easy.  Then when we took it to the shopping center, it was hard to stay for very long.  We started out with me just staying for two seconds.  Over time we built the time up again.  My person knew that stays are going to be very important in my career, so she really took her time when she taught me and now my stays are amazing.  And that's the truth.

DO use the clicker to reinforce attention from the dog.  You can use the clicker to reinforce any behavior and attention is likely the single most important behavior you want your dog to learn.

DON'T use the clicker as an attention getting device; you could be reinforcing behavior you don't want.  Its not a remote control.  Whatever the dog is doing at the instant you click is the behavior that is rewarded and likely to happen again.  So be sure to use it only when the dog is doing something you want to see more of.

DO teach your dog the meaning of a keep going signal (KGS) and use this in training to tell the dog to continue what they are presently doing and that a click/treat is coming.  The best way to use the KGS is to simply use it appropriately in training.  Your dog is really smart and she will figure out what it means.  So if you have been clicking for two steps of heeling and you want more, then give your keep going signal at two steps and then click on the third or fourth step.  Extend this gradually and your dog will learn what it means.  Also, remember that you can use the keep going signal more than once before you click to help your dog understand to keep on trying the present behavior.

DON'T use the clicker as a KGS because your dog will be confused.  Dogs do not understand "context" in all its nuances like humans do, therefore they can only understand one meaning for any one cue (or marker).  If you use the clicker as a marker for "you are right in what you are doing and here's your treat" some of the time and then some of the time as a keep going signal that means "you are getting close to being right", the dog will become confused and not understand which meaning you intend.

DO reinforce behavior you want to see more of, rather than ignore the dog when it is behaving well.  If puppy has been wild all day and has finally settled down to chew its bone, its our human tendency to not do anything that might get puppy stirred up again.  If you do that you are ignoring good behavior.  If you ignore that calm behavior, you may end up with a dog that only calms down when he's exhausted.

DON'T punish misbehavior, you could be reinforcing the behavior with attention.  Dogs thrive on attention; dogs are very social and need our attention.  Remember the song refrain, "Even a bad love is better than no love at all"?   Your dog is likely to think your attention (even a swat) is better than being ignored.

DO manage your environment and training to set your dog up for success and reward.  It is far better to plan life with puppy so that she doesn't have the opportunity to practice bad behavior.  Practice does not make perfect, only perfect practice makes perfect.  If your dog gets to practice bad things, he'll get better at doing bad things.

DON'T "proof" your dog by setting him up to make a mistake. You don't like to be set up to fail and neither does your dog.  "Correction Proofing" only teaches your dog that she can't trust you to be fair.  You want to use "Challenge Proofing" where you set your dog up to be right in gradually increasing challenges.  This will teach your dog how to succeed and be confident.

DO use a high rate of reinforcement and your dog will be eager to train.  In a study of dog trainers, the biggest thing that separated merely good enough trainers from really good trainers was their generosity at dishing out reinforcement.  That doesn't mean you reward any old behavior; rather you carefully observe your dog to find LOTS of good things to reward.

DON'T worry about spoiling your dog with too much reinforcement.  Positive does not equal permissive.  You can set appropriate limits and train our dog to whatever level of excellence you want and still be a kind reward-based trainer.

DO train off leash whenever possible in a safe environment.  Don't let the leash become a physical cue to your dog that now is the only time my person can enforce the rules.  It is far better to teach your dog to want to give you what you want so she can get what she wants.

DON'T use your leash as a training tool.  There is no such thing as a "motivational leash pop" and calling it motivational does not make a leash correction any more fun for the dog.  That's about like calling a face slap "motivational".  It is still a punishment, plain and simple.

DO have a clicker on your person at all times and treats stashed in many places so you can reinforce good behavior whenever it occurs.   Always be prepared to get in a few minutes of training.  The best training sessions are very short.  Fifteen minutes of training per day is a good goal, but you will get better benefit from three 5-minute or five 3-minute sessions per day.  Plus it will be easier for you to squeeze that into your busy life.

DON'T miss impromptu training opportunities.  Training opportunities are present all the time, so don't miss out on using them.  If you are working on loose leash walking and you come upon something enticing on the sidewalk, let go of LLW and use that real life chance to practice "leave it".

Gimme here:  I was walking with my person and found this really great bag of garbage stuff smashed into the pavement.  It was very interesting and I really thought we should investigate it and probably take it home with us.  My person had a different idea.  So we walked by it several times and even around it several times and she gave me treats to ignore it.  As cool as that smelly garbage was, the cheese and steak and chicken I got for ignoring it was even better.  It pays to be clever and smart like me.  I'm just saying.

DO make a list of all of the things your dog finds reinforcing and rank them in order -- remember this is the list for your dog's preferences, not what you think they should be.  There are so many things your dog is willing to work for, if you just know what they are and control access, you can use them to your advantage.

DON'T limit the kinds of things you use for reinforcers.  If my dog wants to go outside, then its a want that I can use to get in a training moment.  Cue a sit or down and reward that behavior by giving the dog what she wants.  You are going to have to open the door anyway, you might as well make a training moment out of it.  All those moments add up.

DO carry a variety of treats so you can use a special treat for a special effort (such as ignoring distraction) or for sudden leaps into brilliant performances.  A special effort or moment of brilliance that goes unrewarded is an opportunity missed.  It may be a long time before it comes again.  Always be ready... even if that means you say your "super reward" word as a marker and follow that with a dash into the house, with pup hot on your heels, to the refrigerator.  Running with you to get the reward can be part of the reward since it adds an element of FUN.

DON'T become predictable to your dog about what you will treat with.  You can use anything that works for your dog.  My young dog loves garlic bread (cut up into little 1/4 crouton sized pieces).  My old dog will do anything for popcorn.  What is your dog's favorite?  Make a potpourri of dog training treats that range from "okay" to "scrumptious".

DO be a benevolent leader who sets fair rules and limitations and teaches the dog what they are and how to live within those rules.  Dogs don't understand our rules and for them its like living in a foreign country with weird customs.  Take the time to teach them how to be a welcomed addition to the family.

DON'T be a malevolent dictator who changes the rules or behaves unpredictably or unfairly to the dog.  You wouldn't like this in a boss or friend.  Why would anyone think a dog would like it?  We all prefer to be around someone who is fair, even if they have high expectations.

DO split goal behaviors into fine criteria and shape from one to the other in a steady progression.  Learning a new behavior is like a set of stairs... its far easier to get to the top one step at a time than it is to try and do it in one giant leap.

DON'T lump behaviors and wait for complete or flawless performances before you reinforce them.  When you get a new job, you will do better and have a smoother transition if you learn the new skills one at a time, using the skills you learned the day before to build on.  The same is true for your dog.

DO plan your training session and how you will shape each behavior so you can reinforce the progression as it occurs.  If you plan ahead how you want to approach the day's training, you will be more likely to make steady progress.  All big projects go better with a little bit of planning.

DON'T miss out on those sudden intuitive leaps skipping several steps, this is the perfect time for a jackpot.  You don't have to be a slave to the plan though, be prepared to change gears and take advantage of the training opportunities that come your way.

DO concentrate on the behavior you want to see your dog do, instead of the things you don't want to see.  Extinction occurs when a behavior goes unrewarded.  So if you reward what you do want and use jackpots to really emphasize the best efforts, your dog will spend her efforts trying to get the jackpots.

DON'T seek to get rid of an undesirable behavior without having a new behavior to replace it with.  Idle paws abhor a vacuum, so always be prepared with a replacement behavior.

Friday, September 30, 2011

Rewards 102

Primary reinforcers
"Primary" just means something that is hardwired into the dog to want; usually related to survival needs, such as food, drink, some touch, and sexual stimulation.

Some breeds have hardwired certain drives that are primary reinforcers, such as vermin for terriers, birds for some hunting breeds, visual stimulation for sight hounds and some sporting breeds, and sheep for herding breeds.  These breeds are often highly sensitive to and easily distracted by these "special" primary reinforcers or things that are similar to them.  For example: movement is distracting to all dogs, but especially distracting to sight hounds and herding breeds.

Some secondary reinforcers can get so strong that they become primary reinforcers, such as tugging.  Dogs aren't born knowing the tugging game.  When they learn to tug it starts as a secondary reinforcer; later it can become primary.

Secondary reinforcers
"Secondary" means something starts out as neutral and had no meaning on its own, but then it is taught or "conditioned" to have value.  Secondary reinforcers depend on you teaching (conditioning) the dog to mentally "pair" it with other primary reinforcers.

For dogs, these conditioned reinforcers can include smiles, praise, attention, clapping, toys, tennis balls, Frisbees, petting and the sound of a clicker.  For humans this includes praise, smiles, thumbs-up gestures, an "A" on homework, and money.

Gimme here:  I love it when my person laughs at me, so I've learned to do a lot of funny things to make her laugh.  Any time I do something that makes her laugh, I remember and do it again and again.  She tells people that I am, "in touch with my inner-clown".   Really I just know when she is laughing, she is happy.  And when my person is happy, I'm happy because I know fun is about to erupt.  Its a great system we have going.

Some conditioned reinforcers, such as the clicker, act as both an "event marker" and a "bridge".  An event marker precisely marks a moment at which the dog is doing a behavior that earns a reward.  It then bridges the gap in time between the moment when the behavior happens and the time when the reinforcement can be given.  Even though you have a bridge, it is still important to deliver the reinforcement as soon as possible or you could also be reinforcing unwanted behavior that happens in the meantime.

Tertiary Reinforcement
"Tertiary" is a special class of reinforcement.  It is when a cue for a known behavior acts as a reward for whatever behavior came before it.  Dogs in agility are rewarded in this way, where cuing the next obstacle reinforces completion of what came before.  It is the opportunity to continue on course and get closer to the reward at the end that makes the next cue reinforcing.

A well trained dog that works for rewards will enjoy working and tertiary reinforcement is highly effective for them.  A dog that is trained with much punishment cannot benefit from tertiary reinforcement.  The idea is that:

      ·   the reward for behavior one is the cue for behavior two
      ·   the reward for behavior two is the cue for behavior three
      ·   the reward for behavior three is the cue for behavior four
      ·   the reward for behavior four is the cue for behavior five
      ·   the reward for behavior five is click/treats

There is a flip side to tertiary reinforcement that you need to know about to keep from unintentionally rewarding undesirable behaviors.

When the dog does something bad, many people will cue it to do something else and then reward the dog for doing that instead.  Since the cue for the new behavior gives the dog the opportunity to earn a reward, it acts as a reward for what the dog did moments before hearing the cue.  I call this I'll Be Bad So I Can Be Good And Get Rewarded syndrome, which is very common with extra smart dogs.

Gimme here:  Just recently I met a horse that was being trained by his person.  When they took a break from training, they came over to where we were.  I barked at the horse to tell him not to come too close (and to tell my person that the horse was close enough).  The lady with the horse asked me to sit and then gave me a treat.  I quickly realized the game possibilities and barked again.  Sure enough, she asked me to sit and then gave me another treat for doing so.   After that I barked a lot and each time she told me "sit" and then rewarded me when I did.  Boyoh was I ever training that lady to give me treats.

Unfortunately, my person knows how that game works and she doesn't want me to bark and she really doesn't want me to be rewarded for barking.  So she made the nice lady stop playing that game with me.  My person explained to her that she should only ask me to sit when I was being quiet, so I didn't play the be-bad-to-be-good game.  After that when I wasn't barking, the nice lady asked me to sit and then rewarded me when I did.  I know that game too, so I was happy to play it.  Then everyone was happy.  My person, nice lady and the horse were happy that there was no more barking; I was happy because I was getting lots of yummy treats.  What a great deal for me.  Life is good if you are trained with rewards.  I'm just saying.

Expand Your Knowledge
There are times when something a dog normally finds rewarding isn't going to be rewarding.

Examples:

      ·   Being warm is rewarding when its chilly, but not when it's
           too hot already.
      ·   Food is a common reward, but when your dog is too full it
           won't work well.
      ·   Dogs that love tennis balls, may lose interest when distracted
           by a female in season.

The same principle works for humans too.

      ·   Being admired by someone is great, but not when that
           someone is creepy.

As important as it is to know what things your dog thinks are rewarding, its also important to know when those rewards aren't rewards at all.  Examine your list of your dog's faves and think about the times when they might not work for you.



Thursday, September 29, 2011

Rewards 101

What is Reinforcement?
Reinforcement, or a reward, is anything your dog will work to get.  If a behavior is happening more often, then it is being rewarded in some way.  Many behaviors are self-rewarding and they will happen more often over time, since the dog simply enjoys doing that behavior or enjoys what it gets from doing that behavior.  An example:  Many dogs enjoy barking and over time will bark more often.  Another example: dogs generally like food and free food even more, so counter-surfing is self-rewarding.  Dogs love to smell good stuff, so even the chance to smell what is on the counter will reward counter-surfing behavior.

Reinforcement is done in two ways - adding a good thing or taking away a bad thing.

If you give your dog a treat when she walks with you on a loose leash that will happen more often.  Likewise if you only move forward when the leash is loose, that behavior will happen more often.  Both are examples of adding a good thing to reward behaviors.  Reward based trainers use this kind of reinforcement most often.

If the dog's behavior makes something "bad" go away (taking away a bad thing) then that behavior tends to happen more often.  Reward based trainers are very careful about using this kind of reinforcement, because you have to add a "bad" thing before you can take it away.  So reward based trainers only use this type of reward in very special cases, such as when the bad thing is already there.

For example:  One special case would be teaching your dog to not fear something, since the scary thing must be present in order to work on becoming less afraid of it.  To reward good behavior in the presence of the scary thing, either the dog can move away from it or the scary thing can move away from the dog (plus give treats at the same time).  Both ways take away stress and scared feelings, so the calm not-afraid behavior is rewarded and will happen more often.

Please be very careful if you try this by yourself.  Working with dog fears and anxiety can be tricky.  Any work dealing with fear or anxiety must be done below the dog's fear threshold.  So, if your efforts seem to be making the matter worse or if the dog can't eat the treats when the scary thing is there, please find a skillful trainer that uses only reward based training to help you.  

Gimme here:  When I was afraid of horses, my person never made me go close to them.  Forcing me to get close to something I was afraid of would only have made me more scared.  On top of that, I would have worried whether my person could be trusted; I need to know my person will keep me safe.

Instead, we watched the horses from far away where I wasn't afraid.  My person also gave me lots of peanut butter to lick up while I was watching the horses.  Peanut butter is the BEST!  I sure love that stuff.  Every time I saw horses I got to eat peanut butter, so I very quickly learned when horses are in sight, peanut butter is on my lips.   It took many months, but now I like horses a lot better.  Just recently I sniffed a horse's nose and let him sniff me.  I'm getting braver all the time because my person doesn't force me to do something I am not ready for.

The one who is receiving the reward (in this case the dog) is the one who "decides" what is rewarding and what is not and how much so.  What one dog loves another may not like at all.  Some dogs like food the most and others are more interested in toys.  Many things that are in the dog's world can be rewards.  If the dog wants it enough to work for it and if you can control access to it, then it is usable as a reward.

My dog Michael was stressed by strange places and wasn't so sure he wanted to do agility.  Every time I tried to get him to do something, he ran off to the edge of the field to pee on bushes.  I knew this was a sign of how worried he was and that he wasn't being disobedient, he was trying to relieve his stress.  I also knew that I couldn't fight him on it if I wanted him to get over it and to one day enjoy agility.  So I put him back on leash and got him to jump one very low jump and then I ran with him to the bushes so he could sniff and pee.  After he'd finished, we went and did two jumps and then ran to the bushes.  We repeated this sequence, sometimes harder and sometimes easier.  It only took a couple of sessions before he realized that this place was okay.  By respecting his needs and using what he wanted (and needed) to do as a reward, he came to love agility.  Learning agility made him more confident and before long he was going to all kinds of new places without any concerns at all.  Michael was halfway to his Masters Agility Championship before we had to retire him because he was too old to jump any more.

I heard of a woman who trained her terrier in agility by using access to nearby trees as rewards.  Her dog didn't want to pee on the trees, he wanted to run around them looking for "little buddies" (squirrels) in the branches.  Instead of fighting that huge distraction, she used it to great advantage.  A dog who can win access to something that distracts them, won't be distracted by it because they know they can have it if they work for it.  Dogs that learn how the training game works are willing to work for stuff.

You and your dog will have different ideas about what is reinforcing.  In this situation, the dog's ideas win.  If you observe your dog closely you will find out what they want.  How many things other than food and toys does your dog want?  Make a list and then be creative and find a way to use those wants to your advantage. 

Gimme here:  I learned on the first day I had my new person that she is very likely to give me what I want if I sit pretty first.  So if I want to go out the door, I sit.  If I want to get out of my crate, I sit.  If I want out of the car, I sit - then she puts my leash on and off we go.  If I want her to play tug with me, I bring a toy and sit.  If I want her to get off the computer and pay attention to me, I sit.  Sometimes you have to nudge them with your nose so they see you are sitting pretty.  Still, most people simply cannot resist a sitting puppy.  It may be most powerful thing you can teach your person.  I'm just saying.

And remember, things that are rewarding to a dog on one day or at one time, may be aversive or neutral at another time.  A dog that has just eaten may not work for treats.  A dog that is really tired, may not work for the chance to play tug with you.  But both of them may work for a belly rub.

Rewards are not as effective when a dog is under stress.  In fact, if a dog is really stressed or fearful, learning cannot happen.  Still you are likely to get more from a stressed dog with rewards than with punishment.  Punishment only increases the anxiety making it unlikely that she can perform or learn.