High schoolers from George Mason programs present research at MIT

In This Story

Mihai Boicu, an associate professor in the Department of Information Sciences and Technology, mentors undergraduate and high school students in two programs at George Mason University, the Aspiring Scientists Summer Internship Program (ASSIP) led by Amanda Haymond in the College of Science and the AI4Defense Scholars Program led by Kamaljeet Sanghera, the executive director of the Institute for Digital InnovAtion (IDIA).

In fall 2024, three student teams Boicu mentored will present their work at the Massachusetts Institute of Technology Undergraduate Research Technology Conference (MIT URTC). The teams, largely comprising high-achieving high school students, will present two papers from ASSIP and one poster presentation from AI4Defense.

MIT URTC is a collaborative effort between MIT undergraduates and the Institute of Electrical and Electronics and Engineers (IEEE). The internationally renowned conference for undergraduate research that takes place this year in mid-October in Cambridge, Massachusetts.

The teams’ projects are described below:

Technical Paper: Assessing the Consistency of Open-Source Large Language Models for Algorithm Evaluation

This study investigates the grading consistency of four open-source large language models (LLMs) on rubric-based assessments of open-ended questions. Among the models, Anthropic Claude demonstrated the highest consistency, while Microsoft Copilot performed the least consistently. The "completeness" rubric category had the lowest variation in scores, making it the most consistently graded category.

Team members for this project represent The Potomac School; the Thomas Jefferson High School for Science and Technology; the University of Illinois Urbana-Champaign; University of Southern California.

“I was ecstatic to see that our paper was accepted. It was an amazing experience to present at MIT URTC — especially as a high school student amongst university students’ research. Preparing for the conference was a lengthy process, but it taught me a lot; I created a presentation and practiced countless times to make sure it would all go smoothly.” - Ava Moazzez, The Potomac School

Technical Paper: Quantitative Analysis of Rubric-based Feedback Received from Claude 3.5 Sonnet on Mathematical Programming Problems

This study evaluates the effectiveness of the LLM Claude 3.5 Sonnet as a grader for Python programming assignments. Four researchers solved five programming problems, received feedback based on a 22-criteria rubric, and resubmitted their work. The results showed a mean score improvement of 17.5 points, with the greatest enhancements in time complexity, efficiency, and edge case handling, highlighting the model’s potential for improving code quality.

Team members for this project represent John F. Kennedy Memorial High School; the Thomas Jefferson High School for Science and Technology; Chantilly High School; Freedom High School and the Academies of Loudoun; Mission San Jose High School; and Beverly Hills High School.

"Every part of the ASSIP experience, from brainstorming ideas to writing an abstract, was very educational and informative. Overall, the program taught me a lot about the research process and prepared me well to present at the MIT URTC Conference" - Rhea Nirmal, Freedom High School

Poster: Testing accuracy of integration of NLP application (ReqFusion) for Air Operation Command

This project addresses the challenge faced by the Air Operations Command (AOC) in managing outdated and new requirements. Using AI-powered tool ReqFusion the team applied Natural Language Processing techniques to integrate legacy and current requirements and improve operational alignment. While the tool demonstrated strong performance in key areas like strategic resource allocation, some limitations in data processing and the clarity of complex requirements indicate room for further improvements in future iterations.

Team members for this project represent Independence High School; Rock Ridge High School; Lightridge High School; and the Thomas Jefferson High School for Science and Technology. This team was mentored by both Boicu and Sanghera.

"Participating in the ASSIP program and presenting our work at the MIT URTC was a transformative experience. Leading my team in developing the ReqFusion system, we tackled real-world challenges for the Air Force, combining AI and NLP technologies to integrate complex requirements. The opportunity to present our research at MIT, share our insights, and receive acceptance for our poster was a proud moment for the team, validating the months of hard work and collaboration." - Vivek Kakarla, Independence High School

Topics

Aspiring Scientists Summer Internship Program (ASSIP)

Institute for Digital InnovAtion (IDIA)