Thursday, April 30, 2015
The Need for Test Data
The vendor said that his company needed access to large amounts of realistic computing evidence to test and refine their product and service. For example, if a vendor develops software that inspects network traffic, it's important to have realistic network traffic on hand. The same is true of software that works on the endpoint, or on application logs.
Nothing in the lab is quite the same as what one finds in the wild. If vendors create products that work well in the lab but fail in production, no one wins. The same is true for those who conduct research, either as coders or academics.
When I asked vendors about their challenges, I was looking for issues that might meet the criteria of Allan Friedman's new project, as reported in the Federal Register: Stakeholder Engagement on Cybersecurity in the Digital Ecosystem. Allan's work at the Department of Commerce seeks "substantive cybersecurity issues that affect the digital ecosystem and digital economic growth where broad consensus, coordinated action, and the development of best practices could substantially improve security for organizations and consumers."
I don't know if "realistic computing evidence" counts, but perhaps others have ideas that are helpful?