In This Story
The software running the world—from our laptops to our cars to our refrigerators—is remarkably complex and requires a tremendous amount of testing. Testing is an onerous, consuming part of bringing a product to market, with a study from Microsoft finding that developers spend up to 22 percent of their workdays testing and debugging.
Wing Lam, an assistant professor in the George Mason University Computer Science Department, recently received a National Science Foundation (NSF) CAREER award and nearly $620,000 to develop more efficient and less expensive tests.
“Every time developers make a reasonable code change, they go through a process called regression testing,” said Lam. “The point is to run tests that have been developed over time to make sure developers’ recent code changes didn't break existing functionality. That's where the majority of software testing is typically centered.”
New code may include human error, such as introducing new logic that conflicts with the existing code, which today is incredibly sophisticated. Lam aims to make testing more efficient, while addressing something known as a “flaky test,” which creates non-deterministic results, failing and passing even when there have been no code changes.
Something as benign as the time a test is run can result in a flaky result. Lam said, “We know that if you're building your code around 5 p.m., your tests may fail more often than if you were to build it early in the morning. The company servers on which the tests are run may be physically in Europe; when it's 5 p.m. on the U.S. East Coast, it can be midnight over there, so if you start running the test at 11:59 p.m. there and it ends after midnight, you may have some flaky test results.”
Efficiency and controlling costs are key to Lam because the industry is in such desperate need of it. He said, “In 2016, Microsoft found that the cost for developers to inspect test results in-and-of-itself can cost millions of dollars for a single product. When developers are wasting time looking at results not related to their recent code changes, that's a waste of their time.”
Looking down the road, Lam hopes to tackle problems related to non-determinism in software beyond just software testing, expanding first into non-determinism problems in software engineering and then into other domains, such as AI and security.
The CAREER award is reserved for the nation’s most talented up-and-coming researchers. From the NSF website: “The Faculty Early Career Development (CAREER) Program offers NSF’s most prestigious award in support of early-career faculty who have the potential to serve as academic role models in research and education and to lead advances in the mission of their department or organization.”