Company
Date Published
Author
Ryan Waldorf
Word count
3805
Language
English
Hacker News points
None

Summary

The author of this text is a developer at MongoDB who experimented with conformance checking, a technique to ensure that the implementation matches the formal specification written in TLA+. The author and their colleagues tried two techniques: trace-checking and test-case generation. Trace-checking involves generating execution traces from the implementation and checking if they conform to the spec, while test-case generation involves creating tests based on the spec and checking if the implementation passes them. The authors' experiments with conformance checking were initially unsuccessful due to difficulties in snapshotting the state of a multithreaded program, discrepancies between the spec and implementation, and limitations of existing tools. However, they learned valuable lessons about the importance of ensuring that the spec and implementation conform from the start, addressing discrepancies promptly, and using more advanced techniques such as model checking and specification-level state exploration. The authors also note that conformance checking can be a powerful tool for catching bugs and verifying correctness in distributed systems, but it requires careful planning and execution. They highlight several research prototypes and papers that have improved conformance checking techniques, including the development of new TLA+ specs, test-case generation tools, and trace-checking features in TLC.