Instructions:Essay instructionsESSAY – SOFTWARE METRICS
Instructions for Spring 2016
In the essay, you will describe one (1) metric. I recommend that you avoid any metric that we’ve spent a lot of time on in class. If your essay primarily parrots back material that we covered in class, you will get very little credit for that work.
1 Don’t use a metric that more than one other student is writing about. You are welcome to share notes with other students, but if two students submit essentially the same research, the grades of both will suffer.
2 You will post your essay in a forum visible to all students. I suggest that you look over other students’ work to get a better idea of the range of work that can be done. However, in essay 2, don’t choose a topic that has an essay 1 written about it unless that essay 1 is so weak that you can’t really learn anything from it. Get permission from me before you use a topic that had an essay 1 written about it.
3 When you choose a metric, apply it to code that you know. Change the code and see the effects of the changes on the metric. In the essay, tell me what you did and what results you obtained. If you can’t actually calculate the metric using your code, find a different metric.
4 A literature review involves several papers / dissertations / books / sources. “Four” is not “several”. I will be surprised if you can do a good job on this paper with fewer than 20. If you can’t find 20 relevant sources, use a different metric.
5 Organize your literature review in terms of the list below. Each item in the list should be a topic heading in your essay. For each topic, present the relevant information that you find in the software engineering literature.
For example, if you were studying defect removal effectiveness, you would probably write about the attribute that it measures (“What is the attribute?”) by referring to papers that try to describe the attribute that it measures.
6 Work through each topic and subtopic. If you can’t find a certain type of information in the literature, say so. Then:
◦ describe the search that you used that didn’t turn up the information. Describe the search specifically – what search terms did you use and what libraries/databases?
◦ Then tell me what you think the answer is or should be
I will base part of your grade on how well you deal with failed searches (how good was your search, how reasonable was your speculation).
7 Don’t be shy about evaluating the credibility, completeness, or other aspects of the quality of the data or the discussions that you find. Be skeptical (keep Leprechauns in mind) but don’t dismiss things unfairly or without reason.
8 Statements of fact need references. A statement of fact is something that can be shown to be true or false. There is (or is not) evidence to support it. If you want to say something about a topic, you have a basis for that. What’s your basis? Tell me your reference.
9 Statements of opinion need references if you got the opinion from someone else, or if you are saying that person X holds this opinion.
10 Anything that is a quote must be in quotation marks. Anything that is a close paraphrase should be turned back into the quote and put in quotation marks. Remember, unattributed quotation will be treated as plagiarism.
HEADINGS OF THE ESSAY
Note: anything in parentheses doesn’t need to be in the actual heading.
Note: The Day 1 lecture describes these headings in a little more detail. If you are confused, look at that slide or post a question in the essay-questions discussion forum.
Overview description of the metric
The core constructs of the metric
• What is the attribute
• Other constructs involved in the measurement
Operations
• Application of the metric to code
• What are the instruments
• How do you take the readings
• What is the scale of the attribute
• What is the scale of the readings
• Commensurability across projects? Companies?
Construct Validity
• Does this metric measure the attribute?
• What model allows us to map this measure to that attribute?
• Key threats to construct validity associated with this metric?
Operational validity
• (Your heading is “Operational validity”. There are no subheadings. Describe the results of research on this metric’s operational validity, including research on common threats to generalization validity of this metric or metrics like it. How well has this been studied and what conclusions do the data support?)
Generalization validity
• (Your heading is “Generalization validity”. There are no subheadings. Describe the results of research on this metric’s generalization validity, including research on common threats to generalization validity of this metric or metrics like it. How well has this been studied and what conclusions do the data support?)
Sources of error
• Directional error (bias)
• Variance of the underlying attribute
• Variance of the measurements
• Biases resulting from intended use or potential consequences
• Biases resulting from intended scope of data or of use
Natural & foreseeable side effects (potential mischief)
• (Your heading can be “Natural & foreseeable side effects” or “Potential mischief”. There are no required subheadings. Describe dysfunction or measurement distortions that are likely when someone uses this metric.)
Grading structure for the essay
This chart shows how I grade the essay.
The chart starts from the assumption that the essay topic is relevant. If you submit a paper that is not clearly about software metrics (for example, a paper on computer hardware reliability), I will immediately set the grade to zero.