Overview
The objective of Stage 2 is to complete the analysis or model building to solve the problem you defined in Stage 1. Activities include: (1) defining an experiment setup; (2) implementing your proposed approach; and (3) evaluating and analysing your approach.
Report (20 marks)
Your report, no more than four pages in length, should describe the experimental setup, approach and results:
Section 1: Setup (4 points)
State your research question(s). State null and alternative hypotheses.
Describe how you will quantify reliability e.g. significance testing, confidence intervals. If appropriate, describe how you will measure effectiveness e.g. regression r-square, clustering V-measure, classification f1-score.
Identify datasets and the data you derived from them.
Section 2: Approach (6 points)
Describe your proposed model and any benchmark models for comparison:
Machine learning projects could focus on e.g. learning techniques, features, model
tuning, parameter selection
Analysis projects could focus on e.g. how the analysis will answer your research questions, operational variables for analysis, method of analysis
Section 3: Results (6 points)
Summarise results, reliability and comparison to any benchmark analyses or models.
Critically analyse results e.g. limitations of data, setup or approach, characteristic errors, possible improvements
Section 4: Conclusion (4 points)
Conclude with what you have learned from this study which would improve yourself as a data analyst. Would you recommend this as a solution to your problem? Provide reasons.
The report is worth 20% of the overall mark for INFO3406. It will be marked on clarity of research question, evaluation framework, approach/implementation, discussion of results and quality of analysis. The report should use the high level headings above – setup, approach, results and conclusion. It should use line spacing of at least 1.15 and body font size of at least 10pt.
Report Marking Criteria
Marks
Setup – Clarity and Specificity of Research Question(s)
0 – 2
Setup – Quantification of Reliability
0 – 2
Approach – Correctness and Completeness
0 – 6
Results – Presentation
0 – 3
Results – Critical Analysis
0 – 3
Conclusion – Conciseness and Recommendations
0 – 4
TOTAL
20
Presentation (during normal tutorial of Week 12)
(7 marks)
The presentation should comprise four to six slides e.g.:
The first stating your questions, hypotheses, reliability quantification and data
The second-third describing your approach and experiment setup;
The fourth summarising results, analysis and discussion.
The presentation is worth 7% of the overall mark for INFO3406. It will be marked on content, style and timing. A four-minute time limit will be strictly enforced. Please practise!
Demonstration Marking Criteria
Marks
Content
0 – 3
Style
0 – 2
Timing
0 – 2
TOTAL
7
Useful style guidelines:
http://sydney.edu.au/engineering/it/courses/info1903/lec31/31_slides_1pp.pdf
