The Texas Sharpshooter Fallacy

I wrote previously on the subject of “Confirmation Bias”. Sometimes it is difficult to discover the answer to a technical problem because the person bringing you the problem has a hitch, assumption, or faulty logic step in their diagnostic process. Often people draw conclusions from spotty evidence. For instance a customer shows me a receiver and say’s “This receiver is bad.” I ask: “Why do you think it is bad? The answer almost always comes back something like “I plugged it in and it does not work.” The person is saying from that one test or measurement they have drawn a conclusion. It seems reasonable doesn’t it? But, really it’s pretty silly when you think about all the things that can cause an RX not to respond to a TX. You see, the real and only conclusion you can draw from the customers test is this. “In one trial, the RX produced no apparent response.” That is quite a bit different from “This receiver is bad.” Understanding the difference in those two conclusions is why some people are good at diagnostics and others are not. To be good at figuring out a problem, you are greatly advantaged by not making any assumptions or broad conclusions.

In the case of a receiver, lets go over many measurements and tests that you might perform to decide if it is in fact “Bad”.

1. Has it ever functioned successfully?
2. Does it really match the Transmitter? (is it talking the right language DSMII vs DSMX or PCM vs FM vs AM and etc…)
3. Is it on the same channel? (in the case of non-2.4ghz gear)
4. Besides looking at the stickers, did you actually look at the tags on the TX and RX xtal?
5. Is the shift the same? For example, a positive shift JR TX is never going to drive a negative shift Hitec or Futaba RX.
6. Have you driven the servo you used on the tested RX with a servo tester to make sure it actually wiggles?
7. Have you load tested the battery your driving the RX with to see that it is high enough to actually turn on an RX?
8. Are you using a switch between the battery and RX? Plug the battery in directly so your not actually testing if the switch is good.
9. Can you demonstrate the TX driving another RX to establish that your testing with a working TX? One might complain, “I flew it a week ago!” However, we’re not testing the troubled RX a week ago, we’re testing it now. 😉
10. Have you plugged a voltmeter into an empty servo port to see if there really is voltage finding it’s way to the RX?
11. Is the crystal really fitting tightly in the socket or is it loose and wobbly?
12. Does the TX have the capability of being on for programming without broadcasting?
13. Is the meter on the TX a voltmeter or RF Output indicator? What does it say?

I’m sure a sharp thinker can come up with some more things to consider. Many of the things above we’ve found at one time or another to be the cause of a non-responsive RX. Assume nothing.

Recently we had an A123 RX pack returned by a customer. He said it tested poorly, only a few hundred mah. The customer appeared to be correct, it was testing bad after several charge/discharge cycles on our bench. And, the charger would increase in voltage rapidly when we applied charge current. Strange. However, even after several trials, a good mechanic still hasn’t drawn any conclusion. He may be moving towards condemning the battery but all tests were not complete. He cut the shrink off the pack. The tabs all looked fine. He re soldered the tabs anyway just in case there was an unseen cold joint. Note: He had originally built the pack, but without emotion, he redid his original work anyway. Many people fail at this step because “they couldn’t possibly have done anything wrong.” (yea right!). The pack was cycled again with the same poor result. Now, finding a bad battery pack is rare, exceedingly rare. We know this to be true from many years of experience. So, we keep looking. I examined the pack under magnification (even though it had been re soldered by a respected pro) and all looked good. I looked at the plug under magnification and found a thin transparent film on the plastic shell. The more I looked, the more I saw this film all over the shell. Is this paint? We decided to solder a second lead onto the pack and test again. The pack tested good. What was the problem you wonder? We can only conclude the film on the plug was thin CA the customer had somehow accidentally allowed to come into contact with the plug. It had a high resistance because one or more connector pins was evidently coated in glue. After replacing this plug, the apparently bad battery pack was proven that it was always as good as new.

So, if a battery pack fails a discharge test or an RX fails to respond, is it bad?

To read about The Texas Sharp Shooter Fallacy, check out this wiki link. Reading it is what inspired me to write today’s article. It describes in somewhat technical language a common way to foul up a test.
“Texas Sharpshooter Fallicy” Wiki Link.

Share
This entry was posted in Blog Entries, How To?, Philosophy, Videos and tagged , , , , , , , , , , , . Bookmark the permalink.