Good Evening! I haven’t blogged for a while so I decided that a good comeback post would be about an idea I’ve had for a web application.
Need
One of the problems I have come across quite a bit in my downloading and using of software (primarily open-source, but also some closed-source) is that, typically, the people who test the software before releasing it into “the wild” are the very same people who are developing that application. You might think this would be a good thing, (if the developers are the ones writing it, they must now how to test it best, right?) but it turns out, according to many psychologists, that when we’re looking for something we can find it pretty easily but if we aren’t looking for it we’ll often miss very obvious things.
In summary, I would say that taking on the paradigm of the user will yield far greater discovery of bugs and inconsistencies than taking on the paradigm of the developer.
Approach
Many application developers can have a difficult time shifting to the user paradigm in order to test their own software because they are so familiar with what it took to get it to where it is and what’s going on behind the scenes (almost as a parent often does not notice their own flaws in their child); this is something that I have learned working for P3 North America, a company that does professional testing. I find, however, that when it comes to using and testing other people’s software it’s an entirely different story.
Thus, my idea is to create a website dedicated to the testing of other people’s software. I believe this would be carried out very well in the style of a time-share. For example, I could post the project I blogged about previously (the Google Calendar Windows Task Connector) and then someone could post their project and we could trade testing time (an hour for an hour) on each other’s projects. This idea could be expanded a bit further to users “earning” hours for the time they contribute and “spending” those hours on their own application.
Benefit
The benefits here are fairly straightforward :
- You, as the software developer, spend time testing other applications rather than your own which gives you a fresh perspective
- You can find more bugs (and usability problems) when looking at others’ programs which is more efficient than testing your own app
- Users have an incentive to test applications and submit bug reports because other users will likely test their applications and do the same for them
Competition
Many commercial software development companies spend hundreds of thousands testing software in preparation for the market (and some still don’t get it quite right), but open-source software developers need to find alternatives to this costly practice. Here are some common methods for testing:
- Test the code yourself and find bugs on your own
- Post the code to Sourceforge or Google Code, publicize your project (i.e. blog about it :P), and hope that people will try the software and submit error reports
- Pay someone to test the program for you (or hold focus groups to ask about usability)
Future
In the future, I would like to add features such as:
- Integration with open-source development and hosting sites
- Recommending projects based on those you’ve already tested
- Rating of testing and fraud detection
- Option for bigger-budget developers to “buy hours”
So, if you like the idea I’ve presented or have anything to add here, please leave a comment. I may be prototyping this in the future. Thanks!
i like the usage of NABC 🙂
also, sign me up!
My response is definitely opinionated based on experience in US commercial sector, so take that into account.
I think your observations are right on. Developers who write software have no clue how to test it..coming from the background of someone who develops software for consumer use targeting all ages.
I’ve had luck in writing test apps that randomly behave to manipulate the software I have written to route out bugs in my own work. Sounds ridiculous right? Unless you’re in an experienced engineering or software house I don’t think management even understands software or why it needs to be well planned in advance then tested.
In America anyway, I really believe the majority of software development takes place inside of businesses. The tough part about your model is that your inherently extending a large amount of trust onto unknown individuals who are testing your software. Major companies such as Microsoft have millions of dollars invested in the secrecy of that product until they release it, because they want to have as much control as they can over the marketing of that product and how information about it is released to the public.
I work in a small company, but our clients would get extremely upset if I were allowing 3rd parties to use their custom software; which in all honesty they were planning to make a lot of money from and planned on enhancing their brand with it.
I do think that open source could maybe benefit from your idea though, but open source developers in the US at least are doing it because they love the work they are doing. I don’t think they are very well funded, but barely survive on jobs that come in based on their open source product. Well; we won’t know unless there is a service like this to prove or deny your claims.
Neat article.
Wow! Thanks for the solid reply. You have a really good perspective. I guess in reply to your post, the app is targeted primarily at Open-Source software development, but I would also like to monetize the site (without just asking for donations). Maybe I could build in a level of Non-Disclosure with a default NDA the company could use, or have an option to include their own.
Thanks for the feedback, this is very valuable.