Edogawa Rampo's definition. "A mystery story is a type of literature that focuses on the amusement derived from solving a complex mystery (usually of the criminal kind) step by step, in a logical way". This definition addresses all the points I find important: the story must feature some kind of mystery to be solved, it is solved in a logical manner and it is fun. The definition also leaves room for variation: like Van Dine I'm always in for a body or two, but if the mystery itself is interesting enough, a lack of dead people certainly doesn't ruin my enjoyment of a mystery story. Logic is also relative and as long as the reasoning is sound within the confines of the story, elements like magic and unknown technology can still result in very good mystery fiction. As you can guess from this definition, I am a fan of puzzle plot mystery stories: "orthodox" mystery stories about giving the reader an intellectual Challenge to solve.
Fair play is often mentioned together with orthodox mystery stories. Fans of the genre have probably heard about Ronald Knox' Decalogue and Van Dine's Twenty Rules For Writing Detective Stories; sets of rules that are intended to make sure any mystery story is actually fair to the reader (i.e. it can be expected from a person with average to above average intellect they can solve the cases based on the clues in the story). Personally, I've never been really convinced by Knox and Van Dine. The only rule that should matter for a puzzle plot mystery story is whether it is fair, and while that is a very subjective criteria, I don't think a hard quotum on twins or hidden passages will help make a mystery story more fair intrinsically.
Ellery Queen is the pinnacle of the fair play mystery story. His early books features Challenges to the Readers: he would simply stop at a certain point in the story, address the reader directly and state that at that point, all the necessary clues to logically deduce the identity of the murderer were given to the reader. This action alone was more important to making detective stories fair than the Thirty Rules above. Here was an author who made it clear that all the puzzle pieces were in place at that point and that the reader wouldn't need to worry anymore about important clues dropping down from the sky at the last second. The reader could turn back the pages and go over everything again just to make sure. The Challenge to the Reader gave the reader a defined range and all the puzzle pieces and that relief of mind appears to me to be of more importance than knowing no Chinaman would appear.
But I think that Queen's Challenge to the Reader on its own wouldn't have been nearly as impressive without the types of clues and hints Queen utilized in his novels and I think that's a topic seldom addressed. His types of clues were perfect for the fair play model, because at the core, it was basic logic and inference. With Carr and Christie, you often need sudden genius insights or psychological analysis, both means which are not particularly 'fair'. With Queen however, the mystery stories are constructed as fair puzzles and if necessary, it's actually possible to solve them with simple determination, rather than a genius mind.
In early Queen novels, solving the crime = identifying the culprit usually boiled down to two basic questions:
1) What are the attributes of the culprit?
2) Who of the suspects answers to all of those attributes?
Sounds simple, and it is actually. Suppose you have a corpse who has been strangled to death by a pair of big hands, and all suspects but one are armless, you have a pretty good idea who the murderer is. Suppose two of them have arms, but you also know the murderer must have entered the crime scene through the little window in the bathroom, you know it was the midget and not the giant man. This method of determining the murderer is very simple in design and absolutely fair. No fantastical ideas or deep psychological analysis. You can just cross off a list of attributes.
Obviously, the trick behind the Queen novels is that is not that easy to figure out what the criminal's attributes are. Let us suppose for this text, that "a clue" is something that came to because of an action, or in-action of the criminal. "Something" is taken in the broad sense of the word, so it can be a phsyical object, but also a state or situation. There's tons of Locked Room Lectures out there, and I've even read essays on all types of mystery stories or typologies of motives in mystery stories, but I still have to read one on clues. So I sorta had to come up with one by myself just now (probably full of gaps, but it'll do for the moment). Clues can roughly be categorized in these four groups:
The clue can be the result of an action, or in-action of the culprit. And these clues were left either intentionally, or unintentionally by the culprit (for convience's sake, we also assume the culprit acts logically and has a sense of self-preservation). Let's try to build on the example above (with the arms) to see how these clues lead to the murderer. Say the ringmaster of a circus was killed in his office room on the second floor of their sleeping/working quarters. Also, because of security measurues we know the murderer must be someone connected to the circus (because that's a lot more convenient).
1) Action/Intentionally: The handmarks left on the neck of the victim. The action is obviously the strangling and in this example, the murderer left the marks as is. We can assume the murderer left the clue=marks as were intentionally: if they really wanted to mask the fact the ringmaster was strangled, they could have cut the neck off and even if suppose there was nothing to cut the head off with at the crime scene, we can assume the murderer was aware of the fact marks were left on the neck and thus left it so knowingly so (even if under different circumstances, they might've cut the head off).
2) Action/Unintentionally: Let's use Holmes' curious incident of the dog in night-time. The ring master kept a dog in his office that hardly does anything dog-like, except for barking at everyone except a select few. The dog was still on the crime scene when the murder was discovered. Nobody heard the dog bark during the time of the murder. Thus the action of the culprit entering the room, plus the fact the dog did not bark, means the dog was on friendly terms with the assailant.
3) Inaction/Intentionally: A button was found lying next to the body, and it did not come from the victim's clothes. The location of the buttom makes it very unlikely the murderer missed it on the way out, so it was left intentionally. Apparently, it is a button from the jackets performers of the circus troupe wear and because of their line of work, it's actually quite common for them to lose buttons all the time. In fact, police investigation showed that everyone with a jacket was missing at least one button (and some of them recent). In this case, the murderer figured that leaving a button wouldn't be enough to identify the murderer (I ignore the possibility of it being a fake clue for convenience's sake).
4) Inaction/Unintentionally: Investigation showed the murderer didn't enter and leave simply through the office door, but through the bathroom door. Which in hind-sight, was a good move, because it just happens that that day a security camera was installed in the corridor. But only a select group of people knew it was being installed. So by avoiding the camera, the culprit also let us know he was in possession of that certain piece of knowledge.
Note that attributes of the other characters aren't clues an sich. Some might be called some form of foreshadowing, but the fact that the midget has arms isn't a clue on its own. It only becomes a clue in combination with the realization that the murderer must have had arms. Note that sometimes, it takes several more logical deductions from the initial clue to reach the correct attribute of the murderer (i.e. a dying message left by the victim could be an Inaction/Unintentionally clue, but you'd need to solve the dying message before it becomes a clue pointing to an attribute).
And the above was a simple example of how clueing and deductions work in Queen(-inspired) novels. There are thus two distinct phases: one is identifying the attributes the culprit must have. The second one is comparing those attributes to those of the suspects and eliminating the suspects until the murderer is left. Note that most of the time, clues tell the reader something concrete about the murderer. The murderer was left-handed! The murderer was color-blind! Or also very popular: the murderer must have known certain facts! The latter in particular is in my opinion a very rewarding type of clue. It usually takes another extra step to deduce the knowledge the murderer must have based on their actions, so when you do realize the murderer must have known about X because they did, or did not, do action Y, it feels very satisfying.
For me, Queen's method especially works because at the core, it's such a simple concept. You don't need Papa Poirot's insight in human psychology. You don't need the genius insight capable of figuring out two impossible crime situations and the identity of a Hollow Man. You just need to determination and a piece of paper to write a little list on. What did the culprit do? What does that tell you about the murderer? Who else has the exact same characteristics? Compare lists, cross off people who don't fit the profile and you're done. You know the murderer is left-handed? Go back through the story and make a note of all the characters to see who is left-handed. The culprit must have been at least two meters high? Check what is noted about the height of each of the characters. In my opinion, these kinds of clues (and method of mystery solving) are about as fair as you can get, as it mostly about combining facts in steps, and doesn't ask for leaps in thinking from the reader.
The thing about Queen novels (and of other people in the Queen school, like Arisugawa Alice and Norizuki Rintarou) is that to determine the culprit, you usually have to combine a lot of these attributes together in order to solve the case, resulting in long chains of deduction. You can guess that with each extra attribute, a story becomes more complex (and boy, these authors can come up with complex plots!), but the building stones of these deduction chains are always of the same variety and while it thus can ask a lot of patience of the reader, these puzzle plots can definitely be solved by going through it one step at a time. A clue of the sort of the slip-of-the-tongue ("Only the murderer would know that!") might be a lot more easier to comprehend, but is not nearly as satisfying as when you managed to combine facts A, B and C to infer X, Y and Z and in extension, the identity of the murderer. To me, this is an extremely fair way to do a whodunit story, because it's essentially a variation of the most basic whodunit possible: One person was killed by a left-handed person. A is left-handed. B is right-handed. There's no discussion about strength of motive here, or about whether someone has the 'mindset' to kill. Just means and opportunity.
On the blog I've often written about the whodunit / guess-the-criminal games at the Kyoto University Mystery Club (where many contemporary Japanese mystery writers originate from). Readers are given the first part of a whodunnit mystery story which ends with a Challenge to the Reader. Participants are challenged to find out who the culprit is before the time limit (usually an hour) and have to explain how they arrived at their answer logically. Most often, these whodunnit scripts follow the method as explained above, for the simple reason it is a very fair way to do a mystery story. Usually, you go through the text hunting for clues the murderer left behind intentionally or not, deduce the set of attributes the murderer must have and then compare that list to the attributes of each of the suspects. While it might sound a bit repetitive, the model has more than enough room for variations and there's nothing that beats that feeling when you've correctly identified all the attributes of the murderer and can logically declare that only X could be the murderer and nobody else.
Obviously, this method of clueing doesn't work well with all mystery stories. Impossible crime stories in particular often ask a bit of daring imagination of the reader, while in the clueing method above, imagination isn't nearly as important as simply being careful in noticing all the attributes. Though it is certainly not impossible to combine the two. Arisugawa's Sweden Kan no Nazo for example features an impossible crime, which is solved by the clueing method above. However, it is a rare example.
Recently, I've been thinking about The Decagon House Murders (for obvious reasons), which is modeled after Christie's And Then There Were None. The latter is a masterpiece of mystery fiction, but is it completely fair? The epilogue refers to three hints, but I'd say that at least two of the three are at the best very vague hints, while the remaining one would still ask of some (uncertain) imagination of the reader. Similarly I've seen people comment that The Decagon House Murders too might be not fairly hinted. True, I too was not sure whether it was completely fair when I first read the book, and unlike And Then There Were None, The Decagon House Murders does not end with a recap of all the important hints. But as I was translating the book, I realized it's probably a lot more carefully hinted than most people (including myself) would suspect at first. Sure, there are no obvious hints like handkerchiefs with initials or deflated balloons lying around, nor are there people who make a fatal slip-of-the-tongue, but The Decagon House Murders's main mystery can be solved by applying the deduction method explained above. By focusing on the actions the murderer took and the knowledge they showed they have, it's absolutely possible to solve the case, as the story is almost surprisingly detailed in its clues (as expected from author Ayatsuji, who wrote a lot of whodunit stories at the Kyoto University Mystery Club). The Decagon House Murders does not follow the method 100% and does ask of a bit of audicity in thinking at one certain point in the deduction chain, but you'd surprised at how much of the truth can be logically deduced by combining and comparing the attributes the murderer must have. The thing is; this is never mentioned within the story itself, so readers are very likely to miss it.
This post has become way too long as is, even though there's still a lot I could talk about: from the way this method of hinting/solving a crime works as a rough guide to mystery story writing to the significance of the 'fake/planted' clue; but I might do that another time. Anyway, clues, good. Any types of clues you particularly like or clues that made an impression on you?