NOTES (talked briefly about this with everyone on the workshop and Zuzel & Nadeem worked on this on Thrusday):
- Time spent in dev infraestructure for developers (not related to work on new features but testing)
- Making the community aware of the importance
- Importance and how much time the community will let us spend on it
- Starting going forward on automating testing
Automated Testing
What do we need (minimum):
Build Server:
Continuous Integration tool setup on it:
- Hudson
- Bamboo (not open source)
- Cruisecontol
SauceLabs for cloud based testing use several free accounts for now for developers.
One free account for the build server
Current Process:
Hudson running on:
- coin.5cat.com:8080
- Checks out changes
- Runs unit tests, and reports to Zuzel
- we do not control the box, and therefore cannot configure ourselves, this limits what we can do
- Current unit test coverage is very very low and therefore Zuzel has to do a lot of manual testing
Hardware/Servers
- Production - Linode - 1 Box (www.p2pu.org)
- Dev - Linode - 1 Box. Currently running etherpad. (alpha.p2pu.org)
- Stats - Linode
- coin.5cat.com:8080 - hudson on it but not controlled by us.
What Next:
- 11/11/11 - Nad to have sent to Zuzel, ubuntu install isntructions for phpunit/sauce extensions etc. AND test framework and examples. [done]
- We develop the scripts locally to run functional tests against Sauce Labs. These can be run locally by developer on his / her machine for now. (wed/16) Nad/Zuzel [done]
- Nad 1/12/11 - Go through setting up lernanta locally for development [done]
- We focus on writing functional tests to cover the main functional areas of the site. (Nad & Zuzel) 3 months.
- We prioritise these over unit tests
- We integrate a code coverage tool ( that we understand ) so we can see the health of the Unit tests, and we can see over 6-12 months that we are adding to that suite. (done code coverage)
- We need to Audit/understand what the difference between the dev and production environments. (Zuzel tbd by 15/11/2011)
- sent dependency versions diff to Nadeem
- We re-purpose our dev/stats box so that it has Hudson installed on it, and automatically checks out the latest trunk, deploys it to alpha.p2pu.org where the functional tests are run against. this then automates the testing of the site. we need to get buy in from John/Philipp
- problem witht he current dev box is that it is running etherpad and as a result memory is a real problem. (3 months)