10
$\begingroup$

Update 2016/09/08: Based on the answer provided by Air (it's long, read toward the middle) and more recent usage on the site, Homework-type questions have been closed with the Unclear What You Are Asking reason.

TL;DR

We do a disservice by offering no guidance to questioners who ask no-effort homework questions that are closed as "off-topic". This can be improved with the addition of a custom close reason that specifically addresses the common issues and links to relevant meta.engineering posts.

My Feelings

I feel like the largest percentage of my close votes are on homework questions that don't show enough effort on the part of the user. Of all of the reasons for closing a question, these are the least well served by the standard options. The users who come and post these questions are the most in need of assistance and the most likely to abandon the site in confusion.

Not every user can be hand-held along the way to learning the site, so the existing site mechanics should be used to give more help than is currently given.

Data

The information below shows the number of questions that have been closed in the last 90 days for each reason. This is all that I have access to.

Homework questions typically get closed under the "off-topic: scope" reason. This is the second largest group, but it probably also contains other questions that aren't homework related.

primarily opinion-based: 10
unclear what you're asking: 65
off-topic - This question does not appear to be about engineering, within the scope defined in the help center.: 60
off-topic - This question belongs on another site in the Stack Exchange network: 14
off-topic - Other (add a comment explaining what is wrong): 8
duplicate: 10

Total: 167

The other reasons as self-explanatory. Out of scope questions are a catch-all, but homework questions are a large portion of the questions that are closed.

Current Information Given

Below is the current banner than is shown to users on closed questions. Please take a second to see how useless this is for homework-type answers. Then click on the help center link to try to find any useful information.

•"This question does not appear to be about engineering, within the scope defined in the help center."
If this question can be reworded to fit the rules in the help center, please edit the question.

The best help is actually when users comment that homework questions are on-topic, but effort is required. Sometimes people even link to the relevant meta post! These are the most helpful, but the information directly contradicts the "off-topic" statement.

Conclusion

I'd like to assume that the information given above shows that we are failing to help our newest users and that we can easily do better. Because of this, I propose that we create a custom close reason that is specifically tailored to no-effort homework questions. This will help guide the questioners by providing relevant information in a single click instead of forcing a new user to dig through our site when they have very little vested interest and may already be confused/mad.

Call to Action

In true meta site style, up and down votes on this question and answers are to serve as votes for or against the proposal and/or answer. Feel free to disagree!

It would be helpful if answers in support of this proposal also include example text and links that should be shown when a question is closed.

Rebuttals to Anticipated Responses

  1. This same thing can be done with comments. - True, but a banner that contains this information is more likely to be read. Also, it is easier for this information to automatically appear when the question is closed versus relying on individual commenters to type similar but different responses every time.

  2. Links can be updated/added to the help center. - This relies on the original poster to click multiple times to find the right information. If they don't find the information, the default close:off-topic text is very misleading, e.g. the question isn't off-topic; it just doesn't show enough effort. This then relies on commenters to clear things up. (See #1)

  3. We should engage the user through comments to encourage them to continue on the site. - Comments are not for chat. Providing this information in comments wastes visible space where more useful comments might appear.

References to Related Meta Questions

What should our position be on homework-type questions?
The Engineering.SE policy on check-my-work questions
Expected level of effort or detail in asking questions?
Do my work for me type questions

| |
$\endgroup$
  • $\begingroup$ MSE background information: What do site moderators need to do to support the new custom question close reasons? $\endgroup$ – Air Jan 28 '16 at 0:27
  • 2
    $\begingroup$ "We do a disservice by offering no guidance to questioners who ask no-effort homework questions...". No, you have it backwards. They are doing us the disservice. We don't owe the lazy students anything, and should deal with them in the most expedient way possible, taking only the affect on the site and the volunteers here into account. $\endgroup$ – Olin Lathrop Jan 28 '16 at 12:25
  • $\begingroup$ @OlinLathrop That is the point. The question is going to be closed regardless. We might as well separate the people that will self-learn from those that will not put forth the effort. As it stands now, no one can be helped from the information given by the system once a question is closed. $\endgroup$ – hazzey Jan 28 '16 at 14:01
  • $\begingroup$ Agreed. It gets old retyping the same "please show your work" statement and providing a link to the meta page. If we have a standard closure for these types of things, it would help new users get the hang of things better. If they end up being the type of people that don't want to do the work, then they won't. $\endgroup$ – grfrazee Jan 28 '16 at 15:22
  • $\begingroup$ The help center is full of detailed articles with clear names like What does it mean if a question is "closed" or "on hold"?. "No one can be helped from the information given by the system" is just not true. I am writing up a more detailed answer that will come tomorrow. $\endgroup$ – Air Jan 28 '16 at 18:53
  • $\begingroup$ Obviously, "tomorrow" has come and gone - I should probably have said "in 6 to 8 hours." :) $\endgroup$ – Air Feb 1 '16 at 16:45
  • $\begingroup$ @Air Actually, if your answer is anything like previous ones, you may be up against the character limit by the time you finish! $\endgroup$ – hazzey Feb 1 '16 at 16:46
  • $\begingroup$ It often takes longer to write less than it does to write more and this is one of those times. $\endgroup$ – Air Feb 6 '16 at 0:29
5
$\begingroup$

No.

I have a lot of thoughts on this issue and I've been trying to figure out the best way to arrange and present them. Really, there are too many things for one answer, so I'm going to try for a more straightforward evaluation of your proposal that spends less time on philosophical objections.

You're right to be concerned about the guidance we provide new users. My gut reaction to your proposal was that it's not necessary or viable but there's no reason we should rely on instinct when there's data to be had. I went through every closed or deleted question that was asked between December 1, 2015 and January 31, 2016 and classified them by hand based on their content. Here are the highlights, excluding nuked spam posts.

Of the 128 closed questions (including deleted):

Of the 17 closed homework questions:

  • 8 (47%) are no-effort homework dumps
    • 5 off-topic
    • 3 unclear
  • 9 (54%) only resemble homework (but may not be)
    • 1 duplicate
    • 2 off-topic
    • 1 too broad
    • 5 unclear

Now let's look at the proposal.

Arguments for a custom close reason

I've given the proposal a fair amount of thought, read it and re-read it since it was posted and I think it can be reduced to four fundamental arguments:

  1. Homework questions that don't show effort are a distinct class of questions that can be treated as homogeneous with respect to closure and guidance.
  2. These questions are a significant problem, representing "a large portion of the questions that are closed."
  3. Close-voters typically use the general off-topic reason to close these questions. (The implication being that this is also correct.)
  4. The guidance provided to new users when we close their questions in this way does a poor job of balancing our desire for the site to be high quality with our desire to welcome new users and help them recover from their mistakes.

The conclusion, if you accept these four arguments, is that we should introduce a special-case close reason for no-effort homework questions. (Keep in mind that these are effectively a limited resource, in that we get three "freebie" off-topic reasons and anything outside of that requires attention and effort from staff.)

1. Homogeneity

I'll concede this one for the sake of brevity; we'll assume that all no-effort homework questions can be dealt with in the same way. As a general rule, this is probably close enough to the truth.

2. Significance

There were 623 non-closed, non-deleted questions asked in December and January. That makes 770 non-spam questions, of which only eight were obvious homework dumps. There's a bit of uncertainty in assessing the nine that I flagged as "possible homework" but it's reasonable to expect the real count of questions that fall into this category is somewhere in the 8–17 range.

I'm not sure if that's the magnitude that you felt you were dealing with or if it's less significant than you thought but it doesn't look like a major issue to me based on frequency, even at the upper bound of 17 questions. We have more than enough users with the experience and site privileges to handle one homework dump every few days, on average, by voting and commenting as normal.

3. The Close Reason

There's a pretty clear split here between off-topic and unclear for choice of close reason. That's not at all surprising, given the guidance in our previous policy discussion:

I'd suggest voting to close as "Unclear what you're asking." It's probably the best close reason as it reflects back "We, the community, don't understand what you don't understand about the problem you asked."

and that, if you browse through some of the history of homework on the network, it's clear that many active users don't want their site to touch homework problems with a 10-foot pole.

It's noteworthy that those questions that are clearly homework are largely the ones we're closing as off-topic. I should point out that I was involved in the closure of only two of the homework questions in this sample: this one, which I closed as unclear despite neither of the two previous close-voters having chosen that option, and this (deleted) one where I was the sole close-voter (and went with too broad).

This seems like a good time to go over some examples:

  • Input Force vs Output Displacement of a System is closed as off-topic. Definitely bad guidance on our part, but not for lack of a better close reason. "Could someone explain how to reason through this?" speaks directly to GlenH7's point that we don't understand what OP doesn't understand.
  • Pivot Point Equations is closed as off-topic, even after OP commented that it wasn't homework. It's a simple matter of applied physics and geometry, and again we're doing a disservice by suggesting a question is "not about engineering." Even worse, a comment directs OP to The Engineering.SE policy on check-my-work questions, which doesn't seem to contain any useful guidance for the situation.
  • Engineering Mechanic problem is deleted after being closed as off-topic. It's a classic homework dump, showing the problem that was assigned with absolutely no other information. Some of our users left comments like if you tell us what you don't understand the community will help, be more specific in what you do not understand and What is giving you trouble? That's good; it tells the author that what they need to do is clarify. It would have been better if users who voted to close had chosen a reason that matched that message.

I agree with Wasabi's answer when it says How do I ask a good question? is the most appropriate Help Center article for these users to read. Fortunately, the banner displayed on questions closed as unclear already points there.

4. Quality vs. Accessibility

This is a perennial topic of debate on larger network sites: How do we get rid of all the crap? versus Why are we so mean/elitist/exclusive?

Maybe it's a foregone conclusion at this point but I do agree that we're providing some pretty unhelpful guidance when we close homework questions as off-topic. It should also be clear by now that there's a much simpler solution than creating a custom reason: Stop closing them as off-topic. Use unclear instead, or too broad when appropriate (deleted).

That's not to say that this is always an obvious or easy conclusion to come to. Many network sites have discussed the same or similar proposals (e.g., Stack Overflow, Computer Science, Math, Physics—the rest are left as an exercise for the obsessive and/or self-loathing reader).

We don't want to be mean or elitist but we do want to be exclusive. We want to exclude people who don't and won't contribute to our overall mission, whether you call that mission "being a good Q&A site" or "building a searchable index of solutions" or "making the world a better place."

Are there users who come here for help and quickly leave in confusion? I hope so! Listen to Joel Spolsky talking about the culture of the network:

...everything about a community either draws people into that community or pushes them away and so many people in web design have been trying to figure out, how do we make a web page that sucks every single person in the known universe in? And when you're trying to get answers to difficult questions, that is the opposite problem. You actually want to drive away as many morons as you possibly can, hopefully as quickly as possible.

I'm not wild about throwing around "morons" like that but I guess when I found something that helps millions of people solve non-trivial problems I'll use the words I like. The takeaway for me is that we want to calibrate our community moderation features—meaning regular voting, closing, reviewing and editing—in such a way that people who are unwilling or unable to figure out how to succeed at asking a question just don't get to participate. I'm less concerned with who's "most in need of assistance" than I am with who can contribute in a positive way.

Big picture philosophy (abbrev.)

We try to judge content on its merits. When someone comes here to ask a question, the fact that they're working on a homework problem is really just context. It's not useful context, like whether your failed connection was welded or bolted; it's irrelevant context. Nobody needs to know whether a question was asked on a Thursday or by an Estonian or using the official Stack Exchange Mobile App for Windows Phone in order to solve the problem that question describes. That type of context shouldn't be a factor when casting close votes.* Breadth, clarity, topicality and subjectivity are the axes of interest and each of these is determined by content. The context of the problem can be valuable content but the context of the question is usually irrelevant.

Yes, it's possible to force homework onto the topicality axis by adding a custom off-topic close reason, but what does that tell the author about how to ask a good question? Isn't it more likely to send one of the following messages?

  • We're here to solve problems but your problems aren't important.
  • We're engineers and homework isn't really engineering.
  • We're experts and you're too young/inexperienced/stupid to engage on our level.

We use close reasons to succinctly illustrate the major failing(s) of a post so that they can be fixed, but the messages above are really just different flavors of go away. When someone comes to us about a homework problem, and we reject them on the grounds that it's homework, they have no remedy; we're not only not telling them how to fix their question, we're telling them they can't fix it. And even new users realize that the difference between a school assignment and a work or hobby project is not particularly relevant to our model of providing useful solutions to reoccurring problems.

Targeting just a subset of homework doesn't fix this problem because it still frames the issue with the author's context in mind. Our policy is not to close things based on a lack of effort. This is generally in line with policies on other sites on the network.

What's worse is that it's not just the author of the bad question who receives this message; it's also other members of the community who happen to see homework questions being shown the door. Including members of the community who participate in commenting and voting but not in meta, who have a less nuanced view of policy (when they're aware of them at all) and will take comments like "homework is out of scope unless you show effort" and rephrase them as worse comments like "homework is off-topic" and "we don't do homework" and eventually just "I'm voting to close this question as off-topic because homework without effort."

That's what I want to avoid, and why I think close reasons tailored specifically for homework dumps do more harm than good.

Epilogue: But we don't like them!

The best catharsis when you don't like a question is the almighty downvote. See also:

Your votes are yours to use how you want, provided each vote is based on the content of the associated item and not on, say, the identity of its author. If you read the content of a question and think, "Ugh, what a lousy homework dump," that's a perfect reason to downvote.


You take it, thermo! Take it and like it!

You can always change your vote later if the question is edited to where you don't think it deserves the downvote. If you don't care to keep tabs on it until then and are worried about leaving a downvote on a question that becomes good, that's fine; don't downvote it, and give up your catharsis. Your choice.

tl;dr:

  • Moderate questions based on their content, not their circumstance. Questions should never be targeted for closure based on extrinsic properties like how much effort the asker put in, whether they are a student or whether they are completing a homework assignment.
  • Close banners will always be the Soup Nazi. Even worse: they're an implacable robot automaton Soup Nazi. No amount of massaging or finesse is going to turn our close banners into the love child of Mr. Feeny and Salman Khan** and that shouldn't even be our goal, because...
  • It's a feature, not a bug! Users abandoning the site who contribute low-quality homework dumps and cannot figure out what makes a good question is entirely desirable.
  • Still, we need to lead by example. We're building a community and our comments on active questions are a strong signal for that process. If you think a struggling user deserves a better outcome, the best way to demonstrate that is not with a form letter, but by engaging them personally and directly. Start with comments, and if it gets too long, take it to chat and/or Meta as appropriate.

* I can only think of one strong counter-example on the network—and that was implemented only after "literally years of the community hating and hammering these questions with close votes and downvotes."

** Some early drafts of this answer featured Bob Ross or Robin Williams. Feel free to substitute warm and fuzzy educators and/or pop culture references of your choosing.

| |
$\endgroup$
  • 1
    $\begingroup$ P.S. Nearly every closed recommendation/shopping question receives at least one comment to explain that we don't do recommendation/shopping questions, and sometimes a link to meta. We get just as many of those questions; do we want to save ourselves some typing and the OP a trip through the Help Center by creating an off-topic reason for those that points them straight to meta? Something to consider. $\endgroup$ – Air Feb 6 '16 at 1:05
3
$\begingroup$

Yes... maybe

The Help Center experience

While I agree with @Air's comment in the OP that the statement that "No one can be helped from the information given by the system" is harsh, I don't think its entirely incorrect. A user can go through the Help Center and find the requirements for on-topicness. However, it's not super easy to find the relevant information.

For instance, the closed statement is:

"This question does not appear to be about engineering, within the scope defined in the help center."
If this question can be reworded to fit the rules in the help center, please edit the question.

If you follow the first link, you're taken to the help center's front page. Here you will probably focus on the Asking category, which currently display the following questions which may be of use to our current curious user:

What types of questions should I avoid asking? isn't of any use for this close reason. The answer lists a few types of questions to avoid, but none of them are in any way related to lack of effort from the asker's side.

What topics can I ask about here? lists characteristics of questions that are on-topic for the site:

  • Identify a specific engineering problem
  • Require expert engineering knowledge to solve
  • Include any relevant drawings, images or references that may be necessary to understand the problem
  • Excite or challenge experienced professionals in the field

"Demonstrable effort by the asker" is not presented here. It does, however, point users to the questions in meta for any further clarifications. I'll get back to this point below.

What does it mean if a question is "closed" or "on hold"? also doesn't help much. It defines an off-topic question as:

This question does not appear to be about engineering within the scope defined by the community. What's on- and off-topic is not always intuitive, so it may be necessary to reword the question to fit this site's scope after reviewing the community guidelines.

The link points to the list of on-topic questions, but it doesn't really show anywhere that "lack of effort" is relevant. If anything, it states that wording might be the issue, which might explain why so many closed questions are edited changing three words in an attempt to see if they are reopened, when the real flaw of the question goes unattended.

So, none of these questions really nailed it for our curious user. Not to be detered, she returns to the Help Center's front page and looks at the Our model section and finds the What kind of behavior is expected of users? question. But that one's mostly a dead end as well, basically just asking people to be nice and honest, nothing about effort.

Still not giving up, our user then goes into Asking: View more, where she finds:

Why do I see a message that my question does not meet quality standards? This one is actually almost there, stating that questions must reach a minimal quality limit defined by the server and suggesting that questions include (among others)

  • A reasonable explanation of what your question is. Add as much detail as you can.
  • Any background research you've tried but wasn't enough to solve your problem.

This is pretty good. It asks for as much detail as possible (which the user might interpret as simply detail on the question, not "show what you've done so far") and for any "research you've tried", which could imply that "work you've done" should also be of value. But this section is explicitly about when the server rejects your question before it's even asked. It starts with "Sometimes users encounter the following message when posting a question" and then continues with "If you see this message, then your question was automatically blocked by the server." A user may very well go "well, this didn't happen to me, so this isn't relevant to my situation" and quit before getting to the list.

How do I ask a good question? is the best question for this matter. In fact, this question is so important that I haven't the slightest clue why it isn't pinned to the Asking section and why its the before-last question in the "Asking > View More" list. I'd rather have this positive question than the negative "What should I avoid" on the front page. It clearly states

Have you thoroughly searched for an answer before asking your question? Sharing your research helps everyone. Tell us what you found and why it didn’t meet your needs. This demonstrates that you’ve taken the time to try to help yourself, it saves us from reiterating obvious answers, and above all, it helps you get a more specific and relevant answer!

Once again, this points to research, not work. I've seen (and voted) a few closed questions where the user doesn't just dump the homework, but states "I've searched quite a bit and tried using X but haven't been able to do it". These are closed because though they show research, they don't tell us exactly where they're stuck. But this is by far the most relevant question for our intrepid user. And, again, it really should be pinned to the Asking section of the Help Center's front page.

Now, for a user to find a clear, explicit mention to homework questions and the need to show your work, they need to go into the questions, as suggested by the "What topics can I ask about here?". This however, is a clumsy approach. At this moment, What should our position be on homework-type questions? is the very last post on the front page. Tomorrow it may be on the second page. A year from now it may be on the fifth. This is hardly an optimal solution. Even then, a new user may not understand how the meta system works and be unsure if seeing some users agree to allow "homework questions showing work" is evidence that that's what closed their question. After all, who says those posts describe the opinion of the entire community?

Should we have a custom close reason?

So, can a homework-user learn via the Help Center that their question was closed due to not showing work? Yes, with some digging and a bit of interpretation. But it's not exactly obvious, especially for someone unfamiliar with the SE model and philosophy.

Would a custom close reason help this user understand why their question was closed? So long as the reason is well-written (something along the lines of "Homework-style questions must show all work done prior to asking and describe precisely where the asker is having trouble. Please edit your question accordingly.", only, you know, well-written), yes, obviously. It will be right there for the user to see: "show your work."

Now, will this improve the community's experience? It improved the asker's by explaining straight-up what's wrong with their question, but what about other users? How often have closed homework questions been edited to satisfaction and reopened? I personally have only seen that happen a handful of times. I would guess that the percentage would increase once more clarity is given to why the question was closed, but don't think it would change by much. So I'm not entirely sure how great the benefit of a custom close reason would be.

Also, there's the issue of how much we prize this sort of question. Do we believe they are an integral part of the site (when well-written) or are they merely "just on-topic enough"? Do we want to make the experience as good as possible for users to ask these sorts of (good-quality) questions, incentivizing more of them? Or would we rather try to disincentivize them ever so slightly, leaving the reasons why some questions are closed opaque? If this site grows, can we even pretend that the proportional number of such questions won't increase as well? Is it worth investing our SE overlords' time creating this close reason for these questions?

I personally think that yes. These questions, while perhaps not requiring the most expert of advice to be answered, add to the corpus of engineering knowledge that may help other engineers-to-be that are probably having difficulty with similar problems. I also, in my ignorance, have no idea how complex it is to create such custom close reasons, but would assume it isn't very difficult. I see basically no downside to creating the custom close reason, though I'm not entirely sure it will have that big of an upside either. That being said, a small upside with no downside is still a positive in my book.

Another solution (instead of or in parallel with this custom close reason) would be to improve the Help Center with specific mentions to such questions or at least to the need for askers to show their work.

| |
$\endgroup$
  • $\begingroup$ I agree completely with your example showing the amount of effort that would be required for someone who really wants to do it right. I also agree that the end result of a custom reason might not be dramatic, but it will be a net positive overall. $\endgroup$ – hazzey Jan 29 '16 at 13:44
  • $\begingroup$ Agreed. While we can't hand-hold every student who asks a homework question, the current "reason" is a disservice to those who did the work, got stumped, and then simply failed to ask in a way that shows they did. $\endgroup$ – Matthias Urlichs Feb 5 '16 at 11:02
1
$\begingroup$

Yes, but 1) only in the worst cases and 2) after the graduation.

An example, what I think as one of the worst cases, can you see here: Simple statics vector analysis problem

The OP didn't even want to rotate his scanned homework upwardly. I can't say any more politely about his behavior.

But, not now. Now the site requires to graduate, and handling even these OPs politely may lead their re-appear to provide more keywords for the google. Thus, my reason is mainly pragmatist about (2).

About (1): network communities tend to inbreed with time, narrow their customs and their self-generated standards tend to differ more and more from the common sense. It leads to a site, where newbies find more and more hard to integrate. It is not an SE-specific problem, it goes so since the first IRC channels. What is SE-specific: (strictly personal opinion) it seems the CMs see this problem and try to do against it, but also their possibilities are limited. The Engineering SE is yet young, but I think extending the possibilities of the closures are more dangerous on the long-term as it seems.

| |
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .