[Hamara-devel] quotes about unit tests and coverage by a python guru

[shirish]

Hi all,
Was reading about testing practises last few days and saw this :-

When did you start using Test Driven Development and why?

I learned unit testing and TDD at school where teachers forced me to 
learn Java, and I hated it. The frameworks looked complicated, and I had 
the impression I was losing my time. Which I actually was, since I was 
writing disposable programs – that's the only thing you do at school.

Years later, when I started to write real and bigger programs (e.g. 
rebuildd), I quickly ended up fixing bugs… I already fixed. That 
recalled me about unit tests and that it may be a good idea to start 
using them to stop fixing the same things over and over again.

For a few years, I wrote less Python and more C code and Lua (for the 
awesome window manager), and I didn't use any testing. I probably lost 
hundreds of hours testing manually and fixing regressions – that was a 
good lesson. Though I had good excuses at that time – it is/was way 
harder to do testing in C/Lua than in Python.

Since that period, I have never stopped writing "tests". When I started 
to hack on OpenStack, the project was adopting a "no test? no merge!" 
policy due to the high number of regressions it had during the first 
releases.

I honestly don't think I could work on any project that does not have – 
at least a minimal – test coverage. It's impossible to hack efficiently 
on a code base that you're not able to test in just a simple command. 
It's also a real problem for new comers in the open source world. When 
they are no test, you can hack something and send a patch, and get a 
"you broke this" in response. Nowadays, this kind of response sounds 
unacceptable to me: if there is no test, then I didn't break anything!

In the end, it's just too much frustration to work on non tested 
projects as I demonstrated in my study of whisper source code.
What do you think to be the most often seen pitfalls of TDD and how to 
avoid them best?

The biggest problems are when and at what rate writing tests.

On one hand, some people starts to write too precise tests way too soon. 
Doing that slows you down, especially when you are prototyping some idea 
or concept you just had. That does not mean that you should not do test 
at all, but you should probably start with a light coverage, until you 
are pretty sure that you're not going to rip every thing and start over. 
On the other hand, some people postpone writing tests for ever, and end 
up with no test all or a too thin layer of test. Which makes the project 
with a pretty low coverage.

Basically, your test coverage should reflect the state of your project. 
If it's just starting, you should build a thin layer of test so you can 
hack it on it easily and remodel it if needed. The more your project 
grow, the more you should make it sold and lay more tests.

Having too detailed tests is painful to make the project evolve at the 
start. Having not enough in a big project makes it painful to maintain it.


and more at 
https://julien.danjou.info/blog/2015/interview-software-tests-in-python

While I don't want to share about the developer so much, much of what he 
shares applies not only to python but to our sort of development and 
integration cycle as well, in fact much more as there would be 
constantly moving parts all the time.

We should be looking at having ci.hamaralinux.org now or in very near 
future. Automated testing would save lot of time for us, time we could 
use to build our skills more in the internals rather than re-fixing any 
issues which will flare up now and again.


-- 
Regards,
Shirish Agarwal,
Community Lead,
Hamaralinux.org