Friday, July 22, 2005

FreeBSD Quality

The topic of the quality of FreeBSD has recently appeared in several places. Earlier this week SecurityFocus reported on the results of a study by Coverity. From Coverity's 27 June 2005 press release:

Coverity "released software defect and security vulnerability results for FreeBSD 6.0... [and] found 306 software defects in FreeBSD's 1.2 million lines of code, or an average of 0.25 defects per 1,000 lines of code."

That is interesting, considering they did the study well over a month ago, before 6.0 was even in BETA status. Also:

"FreeBSD security is getting better very quickly - over the course of a year, FreeBSD's code size doubled, while the total number of defects went down by 50%."

The SecurityFocus story made this observation:

"Not all the potential flaws found by analysis tools are security holes. For FreeBSD, while 306 problems were flagged by Coverity's software, only 5 issues could be triggered by user input. The software classified another 12 vulnerabilities as buffer overruns, another potentially serious security issue. The FreeBSD project has analyzed the flaws and fixed the issues."

For some commentary on the study, check out this thread.

Over on the freebsd-stable mailing list, Alexey Yakimovich started a Quality of FreeBSD thread by complaining about ATA errors. I thought Robert Watson's reply provided very useful insights into the problems of operating system development and testing.


Joao Barros said...

Regarding code quality, Coverity's numbers speak very well of FreeBSD's code quality. As a direct result of that quality, the number of security issues is very low as can be checked on this nice Secunia report:
Very nice to see 100% resolved issues :)

Regarding the mail thread: FreeBSD 5 was a revolution in many areas comparing to 4 and as such was not as polished around the edges as people was expecting. As Scott Long stressed many times, FreeBSD 6 release management will be diferent that 5's, having less features, more testing, more refinement.
I participated in that particular thread trying to lobby a problem I reported at current@ which renders FreeBSD 6 unusable doesn't boot) on a server I have.

tweedledeetweedledum said...
This comment has been removed by a blog administrator.