![]() Output callbacks are just Python functions (or callables) that take in a test record and do what you want to do with it.Īt the end of the test run, OpenHTF calls all the output callbacks in turn, passing each one the completed test record from the test run. To stay with our goal of versatility, OpenHTF lets you set an arbitrary series of output callbacks for your test. Whereas someone who is setting up a test station at the tail end of a manufacturing line in the factory will want to upload the test records to some central data server where the data can be aggregated, queried, etc. Someone who’s running test on the bench in the lab might just want to dump the record to a human-readable JSON file on the local filesystem. Once your test finishes, you have a structured record of test execution.ĭifferent teams and users want to do different things with their test records. When the phase is done, all the measurement PASS/FAIL outcomes are rolled up into an overall PASS/FAIL for the phase, which in turn rolls up to an overall PASS/FAIL for the test itself. Then, inside the phase, you set the measurement as you would a normal variable in Python.Īnd OpenHTF takes care of running the validator you’ve declared on the value you set, and determining whether the measurement is a PASS or a FAIL. You can specify validators (numerical limits being the most common) in the declaration. You declare a measurement by decorating your phase with the decorator. PASS/FAIL is determined by the values assigned to the various measurements. Instantiate a test, passing in those phases.Īdd some output callbacks to do something useful with the test record at the end of the test. ![]() Import the framework, define some phases (functions). This is a pretty distilled example of an OpenHTF test script. Platform agnostic in the sense that it works on all 3 major OS’s (Windows, Mac, and Linux), and doesn’t care what hardware you’re testing, whether it’s a phone, a watch, a camera, a sensor, or another computer.ĪCTS (Android Connectivity Test Suite), which is part of the Android Open Source Project.ĬhromeOS Factory, which is part of The Chromium Projects.Īutotest, originally out of Platforms for Linux Kernel testing. Instead, we leave that to each team to independently support with their own Infrastructure engineering team. When it comes to deployment and scheduling, everyone has different needs, and different systems they need to integrate with, so a one-size-fits-all solution to these things would be riddled with compromises. The intention is for test authors to write hardware test logic only, and to ignore other concerns like deployment and scheduling. Over the course of several iterations, it grew to be lightweight, focused, and platform agnostic. Specifically, we wanted to make something that catered to: OpenHTF came from a team that included both hardware engineers and software engineers working closely together, and was written (and rewritten!) with NON-software-engineers in mind as the primary consumers. Main difference between hw and sw testing is fundamentally “what do you trust?” It also makes these tests difficult to run on a manufacturing floor.Ī notable exception, ChromeOS Factory, is designed for manufacturing, but uses a test-on-device paradigm, which wasn’t suitable for Glass. This makes test authoring familiar to software engineers, but much less so to test engineers, especially vendors/CMs. In some cases they even use certain unit test frameworks’ base classes under the hood. Tests written using the other test frameworks often look similar to software unit tests with assert statements frequently used to control pass/fail outcomes. The origins of OpenHTF are in manufacturing (from the Glass team), a space not well covered by these other tools. With the addition of teams such as Android and ChromeOS, new types of hardware testing became necessary.ĭue to the dearth of existing internal tools (and because This Is Google™), each team developed their own internal tools.ĭifferent teams have different requirements, some are running tests from Google’s internal source repo, some need to ship tests to Vendors/CMs.Įven with those teams, different projects are at different development stages, and have different preferred tradeoffs between test infrastructure and development speed.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |