Python - Pytest and ContextLib

Earlier this year, I got a contribution merged to pytest. I was really happy about this as it solved a confusing error message, and was to a project I use and care about.

Software testing is so important. Without it refactoring complex code becomes more stressful and risk-prone than needed.

Noise while testing can reduce the benefits of tests. Be that unclear messages, or confusing details in tracebacks.

When using standard library hurts test readability

The feature I worked on was around asserting raised errors. I understand the complex and important work the Pytest team do, but also sometimes that does not result in the most optimum API, or supporting code. I looked through how the code worked, and set about a rewrite.

Coming from private industry, I get to use newer python features often. I guess this is a privilege I had overlooked. The only thing that was not working as expected or desired in my own assert_raises implementation, was that context manager internals were showing up. I wanted to know why Pytest did not use contextmanager, but also how they suppressed their own wrapper code.

Great code & documentation

It did not take much digging to find that utility code which is generic and not meant to be surfaced can be omitted from tracebacks using a decorator. This is a real credit to the pytest team, their code and documentation. The below was all I needed.

__tracebackhide__ = True

OpenSource libraries

I have not upstreamed my changes to contextlib. I Feel it might even be inappropriate to do so. I think being a good OpenSource community member means sometimes understanding that it is okay that code does not resonate or meet one coders exact standards, or the way I would do things. Changing contextlib to be aware of a testing library is kind of a smell. I would love to find an alternative way to add core contextlib to a blocklist, in a way that meets the runtime requirements of all testing libraries, including pytest.

I wish there were a PEP standard for omitting logic that is tested externally, like contextlib. I am not aware of one, but it might exist. If you know of it. Please let me know!

Takeaways

• OpenSource code needs contributors.
• Error messaging is important.
• Reducing noise is part of improving error messages.
• Standard library can really help shorten code.
• consider all the inputs before suggesting a change.