Final project
Timeline
due Friday, October 27 (Tuesday labs)
due Sunday, October 29 (Thursday labs)
due Tuesday, November 14 (Tuesday labs)
due Thursday, November 16 (Thursday labs)
Round 1 submission (optional) due Friday, December 1
Presentation + Presentation comments
Tuesday, December 5 (Tuesday labs)
Thursday, December 7 (Thursday labs)
Written report due Wednesday, December 13
Reproducibility + organization due Wednesday, December 13
Introduction
TL;DR: Pick a data set and do a regression analysis. That is your final project.
The goal of the final project is for you to use regression analysis to analyze a data set of your own choosing. The data set may already exist or you may collect your own data by scraping the web.
Choose the data based on your group’s interests or work you all have done in other courses or research projects. The goal of this project is for you to demonstrate proficiency in the techniques we have covered in this class (and beyond, if you like!) and apply them to a data set to analyze it in a meaningful way.
All analyses must be done in RStudio using Quarto and GitHub, and your analysis and written report must be reproducible.
Logistics
You will work on the project with your lab groups. The four primary deliverables for the final project are
a written, reproducible report detailing your analysis
a GitHub repository corresponding to your report
slides and an in-person presentation
formal peer review on another team’s work and presentation feedback
Project proposal
- Friday, October 27 (Tuesday labs)
- Sunday, October 29 (Thursday labs)
The purpose of the project proposal is for your team to identify the data set you’re interested in analyzing for the project, do some preliminary exploratory data analysis, and begin to think about a modeling strategy . If you’re unsure where to find data, you can use the list of potential data sources on the Tips + resources page as a starting point. It may also help to think of topics you’re interested in investigating and find data sets on those topics.
The data set must meet the following criteria:
At least 500 observations
At least 10 columns, such that at least 6 of the columns are useful and unique predictor variables.
- e.g., identifier variables such as “name”, “ID number”, etc. are not useful predictor variables.
- e.g., if you have multiple columns with the same information (e.g. “state abbreviation” and “state name”), then they are not unique predictors.
At least one variable that can be identified as a reasonable response variable.
- The response variable can be quantitative or categorical.
A mix of quantitative and categorical variables that can be used as predictors.
May not be data that has previously been used in any course materials, or any derivation of data that has been used in course materials.
Data that are likely violate the independence condition. Therefore, avoid data with repeated measures, data collected over time, etc.
Data sets in which there is no information about how the data were originally collected
Data sets in which there are missing or unclear definitions about the observations and/or variables
Ask a member of the teaching team if you’re unsure whether your data set meets the criteria.
The proposal will include the following sections:
Section 1: Introduction
The introduction section includes
an introduction to the subject matter you’re investigating (citing any relevant literature)
the motivation for your research question (citing any relevant literature)
the primary research question you are interested in exploring
your team’s hypotheses regarding the research question
- This is a narrative about what you think regarding the research question, not formal statistical hypotheses.
Section 2: Data description
The data description section includes
the source of the data set
a description of when and how the data were originally collected (by the original data curator, not necessarily how you found the data)
a description of the observations and general characteristics being measured
Section 3: Initial exploratory data analysis
In this section, you will begin to explore the data. This includes using narrative, visualizations and summary statistics to describe the following:
distribution of the response variable
distributions of one potential quantitative predictor variable and one potential categorical predictor variable
the relationships between the response variable and each of the predictors from the previous step
a potential interaction effect you’re interested in exploring (it doesn’t have to be an interaction with the two predictors from above)
These steps are to help get you started on exploratory data analysis and will not be the complete EDA for the final report. The requirements above are minimum requirements, but your group is welcome to include more at this stage.
In this section, you will also describe any data cleaning you need to do to prepare for modeling, such as imputing missing values, collapsing levels for categorical predictors, creating new variables, summarizing data, etc.
Section 4: Analysis approach
In this section, you will provide a brief overview of your analysis approach. This includes
a description of the response variable and list of all potential predictors
regression model technique (multiple linear regression or logistic regression)
Data dictionary (aka code book)
Submit a data dictionary for all the variables in your data set in the README of the data folder. You do not need to include the data dictionary in the PDF document.
Submission
Write your narrative and analysis for Sections 1 - 4 in the proposal.qmd file. Put the data set and the data dictionary in the data folder.
Submit the PDF of the proposal to Gradescope. Mark all pages of the document.
Grading
The anticipated length, including all graphs, tables, narrative, etc., is 2 -4 pages; it may not exceed 5 pages.
The proposal is worth 15 points and will be graded based on accurately and comprehensively addressing the criteria stated above. Points will be assigned based on a holistic review of the project proposal.
Excellent (14 - 15 points) : All required elements are completed and are accurate. There is a thorough exploration of the data as descrbied above, and the team has demonstrated a careful and thoughtful approach exploring the data and preparing it for analysis. The narrative is written clearly, all tables and visualizations are nicely formatted, and the work would be presentable in a professional setting.
Strong: (11 - 13 points): Requirements are mostly met, but there are some elements that are incomplete or inaccurate. Some minor revision of the work required before team is ready for modeling.
Satisfactory (8 - 10 points): Requirements partially met, but there are some elements that are incomplete and/or inaccurate. Major revision of the work required before team is ready for modeling.
Needs Improvement (7 or fewer points points): Requirements are largely unmet, and there are large elements that are incomplete and/or inaccurate. Substantial revisions of the work required before team is ready for modeling.
Draft report + peer review
The purpose of the draft and peer review is to give you an opportunity to get early feedback on your analysis. Therefore, the draft and peer review will focus primarily on the exploratory data analysis, modeling, and initial interpretations.
Draft report
Draft is due in your project GitHub repo at 9am on
- Tuesday, November 14 (Tuesday labs)
- Thursday, November 16 (Thursday labs)
Write the draft in the written-report.qmd file in your project repo. You do not need to submit the draft on Gradescope.
Below is a brief description of the sections to focus on in the draft:
Introduction and data
This section includes an introduction to the project motivation, data, and research question. Describe the data and definitions of key variables. It should also include some exploratory data analysis. All of the EDA won’t fit in the body of the report, so focus on the EDA for the response variable and a few other interesting variables and relationships.
Methodology
This section includes a brief description of your modeling process. Explain the reasoning for the type of model you’re fitting, predictor variables considered for the model including any interactions. Additionally, show how you arrived at the final model by describing the model selection process, any variable transformations (if needed), and any other relevant considerations that were part of the model fitting process.
Results
In this section, you will output the final model and include a brief discussion of the model assumptions, diagnostics, and any relevant model fit statistics.
This section also includes initial interpretations and conclusions drawn from the model.
Grading
The draft will be graded based on whether there is demonstration of a reasonable attempt at each of the sections described below in the written-report.qmd file in our GitHub repo by the deadline.
Peer review
Peer review comments are due in GitHub at 11:59pm on
- Wednesday, November 15 (Tuesday labs)
- Friday, November 17 (Thursday labs)
Critically reviewing others’ work is a crucial part of the scientific process, and STA 210 is no exception. Each lab team will be assigned two other teams’s projects to review. Each team should push their draft to their GitHub repo by the 9am on the day their lab’s draft is due. The lab that week will be dedicate to the peer review, so your team will have time to review and provide quality feedback to two other teams.
During the peer review process, you will be provided read-only access to your partner teams’ GitHub repos. Provide your review in the form of GitHub issues to your partner team’s GitHub repo using the issue template provided in the repo.
Steps for peer review
Click here to see which project your team is reviewing. You’ll spend about 30 minutes reviewing each project.
When you get to lab, you should have access to the GitHub repos for the teams you’re reviewing. In GitHub, search the repositories for project, and you should see the repos for the projects you’re reviewing. You will be able to read the files in the repo and post issues, but you cannot push changes to the repo. You will have access to the repo until the deadline for the peer review.
For each team you’re reviewing:
Open that team’s repo, read the project draft, and browse the rest of the repo.
Go to the Issues tab in that repo, click on New issue, and click on Get started for the Peer Review issue. Fill out this issue. You will answer the the following questions:
Describe the goal of the project.
Describe the data set used in the project. What are the observations in the data? What is the source of the data? How were the data originally collected?
Consider the exploratory data analysis (EDA). Describe one aspect of the EDA that is effective in helping you understand the data. Provide constructive feedback on how the team might improve the EDA.
Describe the statistical methods and analysis approach used.
Provide constructive feedback on how the team might improve their analysis. Make sure your feedback includes at least one comment on the statistical modeling aspect of the project, but also feel free to comment on aspects beyond the modeling.
What aspect of this project are you most interested in and would like to see highlighted in the presentation?
Provide constructive feedback on any issues with file and/or code organization.
(Optional) Any further comments or feedback?
Grading
The peer review will be graded on the extent to which it comprehensively and constructively addresses the components of the partner team’s report: the research context and motivation, exploratory data analysis, modeling, interpretations, and conclusions.
Written report
Your written report must be completed in the written-report.qmd file and must be reproducible. All team members should contribute to the GitHub repository, with regular meaningful commits.
Before you finalize your write up, make sure the code chunks are not visible and all messages and warnings are suppressed.
You will submit the PDF of your final report on GitHub.
The PDF you submit must match the .qmd in your GitHub repository exactly. The mandatory components of the report are below. You are free to add additional sections as necessary. The report, including tables and visualizations, must be no more than 10 pages long. There is no minimum page requirement; however, you should comprehensively address all of the analysis and report.
Be selective in what you include in your final write-up. The goal is to write a cohesive narrative that demonstrates a thorough and comprehensive analysis rather than explain every step of the analysis.
You are welcome to include an appendix with additional work at the end of the written report document; however, grading will overwhelmingly be based on the content in the main body of the report. You should assume the reader will not see the material in the appendix unless prompted to view it in the main body of the report. The appendix should be neatly formatted and easy for the reader to navigate. It is not included in the 10-page limit.
Introduction and data
This section includes an introduction to the project motivation, data, and research question. Describe the data and definitions of key variables. It should also include some exploratory data analysis. All of the EDA won’t fit in the paper, so focus on the EDA for the response variable and a few other interesting variables and relationships.
Grading criteria
The research question and motivation are clearly stated in the introduction, including citations for the data source and any external research. The data are clearly described, including a description about how the data were originally collected and a concise definition of the variables relevant to understanding the report. The data cleaning process is clearly described, including any decisions made in the process (e.g., creating new variables, removing observations, etc.) The explanatory data analysis helps the reader better understand the observations in the data along with interesting and relevant relationships between the variables. It incorporates appropriate visualizations and summary statistics.
Methodology
This section includes a brief description of your modeling process. Explain the reasoning for the type of model you’re fitting, predictor variables considered for the model including any interactions. Additionally, show how you arrived at the final model by describing the model selection process, interactions considered, variable transformations (if needed), assessment of conditions and diagnostics, and any other relevant considerations that were part of the model fitting process.
Grading criteria
The analysis steps are appropriate for the data and research question. The group used a thorough and careful approach to select the variables in the final model; the approach is clearly described in the report. The model selection process took into account potential interaction effects and addressed any violations in model conditions. If violations of model conditions are still present, there was a reasonable attempt to address the violations based on the course content.
Results
This is where you will output the final model with any relevant model fit statistics, conditions, and diagnostics.
Describe the key results from the model. The goal is not to interpret every single variable in the model but rather to show that you are proficient in using the model output to address the research questions, using the interpretations to support your conclusions. Focus on the variables that help you answer the research question and that provide relevant context for the reader.
Grading criteria
The model fit is clearly assessed, and interesting findings from the model are clearly described. The model conditions and diagnostics are thoroughly and accurately assessed for their model. Interpretations of model coefficients are used to support the key findings and conclusions, rather than merely listing the interpretation of every model coefficient. If the primary modeling objective is prediction, the model’s predictive power is thoroughly assessed.
Discussion + Conclusion
In this section you’ll include a summary of what you have learned about your research question along with statistical arguments supporting your conclusions. In addition, discuss the limitations of your analysis and provide suggestions on ways the analysis could be improved. Any potential issues pertaining to the reliability and validity of your data and appropriateness of the statistical analysis should also be discussed here. Lastly, this section will include ideas for future work.
Grading criteria
Overall conclusions from analysis are clearly described, and the model results are put into the larger context of the subject matter and original research question. There is thoughtful consideration of potential limitations of the data and/or analysis, and ideas for future work are clearly described.
Organization + formatting
This is an assessment of the overall presentation and formatting of the written report.
Grading criteria
The report neatly written and organized with clear section headers and appropriately sized figures with informative labels. Numerical results are displayed with a reasonable number of digits, and all visualizations are neatly formatted and labeled. All citations and links are properly formatted. If there is an appendix, it is reasonably organized and easy for the reader to find relevant information. All code, warnings, and messages are suppressed. The main body of the written report (not including the appendix) is no longer than 10 pages.
Submission
The written report is due on Wednesday, December 13 at 11:59pm.
To submit your report, written-report.qmd and the rendered written-report.pdf to your team’s GitHub repo by the deadline. You will not submit the report on Gradescope.
The version of the report in the repo by Wednesday, December 13 will be the one that is graded.
Round 1 submission (optional)
Friday, December 1 at 11:59pm on GitHub (all teams)
Reports submitted after this date will not receive preliminary feedback.
The Round 1 submission is an opportunity to receive detailed feedback on your analysis and written report before the final submission. Therefore, to make the feedback most useful, you must submit a complete written report to receive feedback. You will also be notified of the grade you would receive at that point. You will have the option to keep the grade (and thus you don’t need to turn in an updated report) or resubmit the written report by the final submission deadline to receive a new grade.
To submit the Round 1 submission:
Push the updated
written-report.qmdandwritten-report.pdfto your GitHub repo.Open an issue with the title “Round 1 Submission”. You can use the template issue in the GitHub repo. Make sure I am tagged in the issue (
@matackett), so I receive an email notification of your Round 1 submission. See Creating an issue from a repository for instructions on opening an issue. Please ask a member of the teaching team for assistance if you need help opening the issue.
Note that this is optional, so there is nograde penalty for not turning in a Round 1 submission. Due to time constraints at the end of the semester, only high-level feedback will be given for the reports submitted at the final written report deadline on December 13.
Presentation
Presentations will take place in class during labs December 5 & 7.
Click here for the presentation order.
In addition to the written report, your team will also do an in-person presentation that summarize and showcase your project. Introduce your research question and data set, showcase visualizations, and discuss the primary conclusions. The presentation should be supported by slides that serve as a brief visual addition to the presentation. The presentation and slides will be graded for content and clarity.
You can create your slides with any software you like (Keynote, PowerPoint, Google Slides, etc.). We recommend choosing an option that’s easy to collaborate with, e.g., Google Slides.
You can also use Quarto to make your slides! While we won’t be covering making slides with Quarto in the class, we would be happy to help you with it in office hours. It’s no different than writing other documents with Quarto, so the learning curve will not be steep!
The presentation must be no longer than 6 minutes. It is fine if the presentation is shorter than 6 minutes, but it cannot exceed 6 minutes due to the limited time during lab.
Every team member is expected to speak in the presentation. Part of the grade will be whether every team member had a meaningful speaking role in the presentation.
Slides
The slide deck should have no more than 6 content slides + 1 title slide. Here is a suggested outline as you think through the slides; you do not have to use this exact format for the 6 slides.
- Title Slide
- Slide 1: Introduce the topic and motivation
- Slide 2: Introduce the data
- Slide 3: Highlights from EDA
- Slide 4: Final model
- Slide 5: Interesting findings from the model
- Slide 6: Conclusions + future work
Grading criteria
The presentation grade will be based on the following criteira:
Content: The group told a unified story using the appropriate regression analysis.
Slides: The presentation slides were organized, included clear and informative visualizations, and were easily readable.
Professionalism: The group’s communication style was clear and professional.
Time Management: Team divided the time well and stayed within the 6 minute time limit, with each team member making a meaning contribution to the presentation. (assessed by the teaching team only).
80% of the presentation grade will be the average of the teaching scores and 20% will be the average of the peer scores.
You can submit the presentation slides in two ways:
Put a PDF of the slides in the
presentationfolder in your team’s GitHub repo.Put the URL to your slides in the
READMEof thepresentationfolder. If you share the URL, please make sure permissions are set so Prof. Tackett can view the slides.
Slides must be submitted by the start of your lab on December 5 or 7. You will not submit the slides on Gradescope.
Presentation comments
Click here to find the teams you’re scoring and a link to the feedback form.
This portion of the project will be assessed individually.
You will provide feedback on two teams’ presentations. You can find your assigned teams and the link to the feedback from here. Please provide all scores and comments by the end of the lab session. There will be a few minutes between each presentation to submit scores.
The grade will be based on submitting the scores and comments for both of your assigned teams by the end of the presentation day (December 5 for Tuesday labs, December 7 for Thursday labs).
Reproducibility + organization
All written work (with exception of presentation slides) should be reproducible, and the GitHub repo should be neatly organized.
The GitHub repo should have the following structure:
README: Short project description and data dictionarywritten-report.qmd&written-report.pdf: Final written reportproposal.qmd&proposal.pdf: Project proposal/data: Folder that contains the data set for the final project.project.Rproj: File specifying the RStudio project/presentation: Folder with the presentation slides or link to slides..gitignore: File that lists all files that are in the local RStudio project but not the GitHub repo/.github: Folder for peer review issue template
Points for reproducibility + organization will be based on the reproducibility of the written report and the organization of the project GitHub repo. The repo should be neatly organized as described above, there should be no extraneous files, all text in the README should be easily readable.
The repo must be ready for grading by Wednesday, December 13 at 11:59pm.
Peer teamwork evaluation
You will be asked to fill out a survey where you rate the contribution and teamwork of each team member by assigning a contribution percentage for each team member. If you are suggesting that an individual did less than half the expected contribution given your team size (e.g., for a team of four students, if a student contributed less than 12.5% of the total effort), please provide some explanation. If any individual gets an average peer score indicating that this was the case, their grade will be assessed accordingly.
Overall grading
The grade breakdown is as follows:
| Total | 100 pts |
|---|---|
| Project proposal | 15 pts |
| Draft report + peer review | 15 pts |
| Presentation | 20 pts |
| Presentation comments | 5 pts |
| Written report | 40 pts |
| Reproducibility + organization | 5 pts |
Grading summary
Grading of the project will take into account the following:
Content - What is the quality of research and/or policy question and relevancy of data to those questions?
Correctness - Are statistical procedures carried out and explained correctly?
Writing and Presentation - What is the quality of the statistical presentation, writing, and explanations?
Creativity and Critical Thought - Is the project carefully thought out? Are the limitations carefully considered? Does it appear that time and effort went into the planning and implementation of the project?
A general breakdown of scoring is as follows:
90%-100%: Outstanding effort. Student understands how to apply all statistical concepts, can put the results into a cogent argument, can identify weaknesses in the argument, and can clearly communicate the results to others.
80%-89%: Good effort. Student understands most of the concepts, puts together an adequate argument, identifies some weaknesses of their argument, and communicates most results clearly to others.
70%-79%: Passing effort. Student has misunderstanding of concepts in several areas, has some trouble putting results together in a cogent argument, and communication of results is sometimes unclear.
60%-69%: Struggling effort. Student is making some effort, but has misunderstanding of many concepts and is unable to put together a cogent argument. Communication of results is unclear.
Below 60%: Student is not making a sufficient effort.
Late work policy
There is no late work accepted on the draft report or presentation. Other components of the project may be accepted up to 48 hours late. A 10% late deduction will apply for each 24-hour period late.
Be sure to turn in your work early to avoid any technological mishaps.