ENABLING IMPARTIALITY: Not every competitor is a perfect 10. Tools like Valid Eval allow judges to base their scores on specific, measurable criteria, for more accurate results. (Image by Getty Images)
The Army gets help from a small business to improve decision-making for technology solutions.
by Stephanie Price and Anna Volkwine
With a winning partnership between the Army and an innovative small business, DOD has a low-barrier, streamlined approach to identify transformative technology solutions. That small business is Valid Eval, and its evaluation tool has given the Army a simple-to-use process to get technological innovations into the hands of Soldiers with greater confidence and greater speed.
In a typical evaluation process for Army prize competitions and proposal reviews, a judge could expect to sit in deliberation for two to three days straight, discussing and negotiating with peers about which technologies could be the next innovations to ensure mission readiness for Soldiers. By partnering with Valid Eval, it not only simplified the process, but it implemented a more efficient, reliable way to make these funding decisions for DOD.
HOW IT BEGAN
Adam Rentschler, co-founder and CEO of Valid Eval, said he’s always been bothered by how innovation is evaluated.
“Everything is driven by experts’ gut instincts. For those being judged, the process is opaque, mysterious and without any feedback loops other than a ‘no thanks,’ ” Rentschler said. “There had to be a better way.”
Rentschler was complaining about this problem to a friend, Todd Reimer, a Ph.D. learning scientist, who began to detail a scientific approach to evaluating complex performances like innovation pitches. Rentschler, Reimer and Kent Hollrah, a user experience expert, joined forces. Together, they developed a prototype for a secure platform to manage complex group evaluations with transparency, efficiency and accountability at the forefront. From there, Valid Eval was born. “One of the things that makes our tool different is that we deliver a consumer-grade user experience to expert evaluators,” Rentschler said.
A secure software-as-a-service platform, Valid Eval is an online evaluation system that helps organizations make and defend tough decisions. It can work on virtually any scale, so companies or organizations can involve as many applicants, experts or judges as they need, while providing defensible, data-driven results and robust reporting tools to help them monitor and measure performance.
In 2018, Rentschler reunited with a former business colleague who worked for the U.S. Air Force and required a tool to provide more robust feedback to companies not selected for its business accelerator competition, the Hyperspace Challenge. One of the biggest areas of improvement Rentschler wanted to address with his tool was the lack of valuable feedback provided to the innovation community. Valid Eval was able to provide the Air Force just that—clear-cut and applicable feedback, delivered in a neatly packaged and easy-to-read format. And now that the Air Force’s front door was open, Valid Eval quickly learned that there were many more opportunities for it to continue doing business across various DOD agencies that are all responsible for having efficient, effective and transparent acquisition processes.
TEST RUN: U.S. Air Force Lt. Col. Richard Tanner, left, 18th Wing vice commander, reviews submissions using the Valid Eval system in 2019 at Kadena Air Base, Japan. The Kadena group was the first Air Force “spark cell” innovation group to use the Valid Eval system. (Photo by Senior Airman Michael Jones, 18th Wing Public Affairs)
GOING GREEN
Valid Eval formed a partnership with the Army in 2019, streamlining the evaluation process for xTechSearch 2, the second xTechSearch competition. This was only the first step in a long-term relationship with the Army and a clear demonstration of how small businesses can provide large-scale solutions for the Army enterprise.
Sponsored by the Assistant Secretary of the Army for Acquisition, Logistics and Technology (ASA(ALT)), the xTechSearch competition was launched in 2018 to identify and award transformative technology solutions, from small, non-defense businesses, to sustain Army priorities and fill critical mission gaps. Through xTechSearch, winners have the opportunity to earn significant cash prizes, to secure potential follow-on contracts and agreements with the Army, and to find unique opportunities to engage with key defense stakeholders through events, mentoring and educational programming.
To determine which businesses can most impact Soldier readiness, each competing organization goes through an extensive judging process. For the xTech Program’s first prize competition, now known as xTechSearch 1, the Army used a government-developed tool that suffered from a lack of recorded feedback and a difficult user experience.
To simplify the review and submission process for both evaluators and participants, the xTech Program adopted the Valid Eval tool for the second iteration, xTechSearch 2, which required more than 130 expert judges to review over 150 applicants. Over the past three years, the xTech Program has grown to 500 judges for xTechSearch 5, which launched in February 2020 and announced the grand-prize winner, Project Owl, on Sept. 1, 2021.
“We’ve made efforts to make this easy and intuitive,” Rentschler said. “Judges are having fun with the process—that helps a great deal.”
This technique arms evaluators and decision-makers with the knowledge to demonstrate mission alignment with the Army and other potential clients. Understanding and meeting government needs are critical components of a business’s success, and providing an easy path to do so propels these businesses to the front of the line when it comes to awarding cash prizes and funding.
“We want to be able to start solicitations as needs emerge from Army senior leaders, and get decisions made on a rapid cycle, rather than the traditional process that can take 12, 18 or 24 months,” said Dr. Matt Willis, director of Army Applied Small Business Innovation Research (SBIR) and Army Prize Competitions in the Office of the Deputy Assistant Secretary of the Army for Research and Technology (DASA (R&T)).
With Valid Eval, the biggest advantage is the elimination of lag time. Instead of manually scoring and tallying results, then filtering them through to each competitor, automation helps to select the best technologies to support Army research and development and acquisitions programs. The sooner new technology can be introduced to Soldiers, the better their performance and reliability, which can ultimately be lifesaving on the front lines.
To date, xTech has used Valid Eval as the judging and scoring mechanism in over 15 competitions and counting. Valid Eval will continue to be a major player in making these research and development and acquisitions decisions better and faster, as well as highlighting a meaningful small-business approach to Army problems.
TOOLING AROUND: Maj. Billy Thomas, left, and Maj. Ryan Fillis, right, used the Valid Eval tool in the xTechSearch 4 competition to evaluate, score and deliberate on science and technology innovations in January 2020. (Photo by xTechSearch)
ANOTHER ARMY DOOR OPENS
Valid Eval is helping the Army make the right technology acquisition decisions faster while meeting critical DOD needs through a partnership with the Army Applied SBIR Program started in April 2021. The Army Applied SBIR Program sponsors research and development for small businesses to provide innovative solutions to meet urgent Army modernization needs. The program also connects Soldiers and other subject matter experts with small businesses to provide insight and mentorship through the Army research and development ecosystem.
To capitalize on small business innovation and respond to the most urgent Army capability needs, the Army Applied SBIR Program issues contract opportunities on an ad hoc, rolling basis to address needs as they emerge—a switch from the legacy SBIR process of releasing topics via pre-determined announcements issued at the DOD level. Valid Eval supports this effort by helping Army Applied SBIR make fast, reliable decisions that are in the best interest of the Army.
“We have a tremendous amount of interest in the solicitations we put out on the street, and we’re trying to bring innovation back to the SBIR program,” said Willis, who has led the Army Applied SBIR Program for nearly two years. “We want to be expeditious in our approach by scouting the tech landscape to determine which technologies respond to not only the Army’s needs, but small business needs as well.”
The program awards topics that clearly align with the Army’s modernization priorities as well as DOD’s key science and technology areas, such as artificial intelligence and machine learning, and advanced materials and communication technologies, which dictate the Army’s investment decisions. To make this happen, the program is shifting from a passive, topic-driven approach to the newly created transition broker team construct, which manages a centralized portfolio and synchronizes among stakeholders to identify the best emerging technical solutions while integrating all program activities.
A crucial step in determining which companies meet these urgent Army priorities is providing unbiased decisions as quickly as possible and giving transparent feedback. Even if companies don’t receive an award, they can understand how they can pivot their proposal or technology to be more viable in the future.
Valid Eval was awarded a new task order on its Phase III indefinite duration, indefinite quantity SBIR contract with the General Services Administration. It also won an Air Force Phase I SBIR in 2018 and secured a Phase II from the Air Force in early 2019.
Valid Eval is now the primary tool for evaluating proposals that small businesses submit to the Army Applied SBIR Program. It allows the program to set clear expectations for how a company will be evaluated, collate responses and reviews from experts, give an unbiased assessment, and provide data-backed recommendation to Army leadership on the best proposals.
“We’re empowering our program managers to make decisions under the best interest of the Army, and have those decisions be supported by a data-driven tool,” Willis said.
CONCLUSION
With technology constantly evolving, the Army must always progress and improve in order to deter and defeat adversaries. Identifying technologies, providing funding, prototyping, and implementing solutions can be a long process, so time-saving tools are a must—and Valid Eval hits the mark for efficiency.
“The shared mission is to make innovation better for the DOD,” Rentschler said. “The really cool thing about our partnership with xTech and SBIR is that they are willing to take some smart risks and do things differently than they’ve done in the past, which is why collaboration is key to these relationships.”
Collaboration with small businesses like Valid Eval, Army prize competitions and other contracting mechanisms are a model for how distinct entities can work together to create innovative processes to bring trust, efficiency and improved performance to the materiel development process, and drive research and development and acquisition programs to find the best technology and partners for government programs.
“We now have a process that makes acquisition better for the DOD—and that’s thanks to a partnership between the federal government and a small business,” Willis said. “These crucial partnerships not only foster, strengthen and encourage the role of small businesses, but also help us modernize our world-class military and transition life-saving technology into the hands of our Soldiers.”
For more information, go to https://www.armysbir.army.mil/.
STEPHANIE PRICE provides contract support to the Army Applied SBIR Office for Booz Allen Hamilton. She holds an M.A. in communication from The Johns Hopkins University and a B.A. in communications studies from West Chester University.
ANNA VOLKWINE provides contract support to the Army xTech Program for Booz Allen Hamilton. She holds a B.A. in communications from Mount St. Mary’s University.
