View previous topic :: View next topic |
Author |
Message |
enovak
Joined: 24 Nov 2021 Posts: 5
|
Unit Test Framework |
Posted: Thu Dec 09, 2021 9:25 am |
|
|
I've been researching implementing some automated unit test in my project. I know that there are a number of different frameworks which are available to use. I was curious if anyone has implemented anything using the CCS compiler. Any input on the best framework to use? Are you executing your tests on target or simulating them? Thanks for the input. |
|
![](templates/subSilver/images/spacer.gif) |
zaz
Joined: 18 Dec 2019 Posts: 2
|
|
Posted: Thu Mar 10, 2022 3:19 am |
|
|
I am also looking for unit test framework for CCS project. |
|
![](templates/subSilver/images/spacer.gif) |
temtronic
Joined: 01 Jul 2010 Posts: 9168 Location: Greensville,Ontario
|
|
Posted: Thu Mar 10, 2022 6:21 am |
|
|
I can't see any UTF or another 'simulator' actually being 100% accurate with using a simple PIC like the 16C84, let alone a modern version, say a 46K22.
The problem I see is that the UTF programmer has no hands on, real world experience with a PIC ? Just HOW does he emulate random cross talk that feeds into the 2nd UART and corrupts the data stream ? Or the effects of a nearby lightning bolt ? |
|
![](templates/subSilver/images/spacer.gif) |
Ttelmah
Joined: 11 Mar 2010 Posts: 19358
|
|
Posted: Thu Mar 10, 2022 6:40 am |
|
|
Dead right.
I have used Criterion to make test values to prove functionality of some
code in the past, but the problem is that instrumentation code is far more
likely to have problems from things like EMP, than from general 'code errors'.
I have yet to see any simulator that I would rate as being more than 99%
accurate, and all will miss 'oddities' with particular chips, that the real
instrument has to handle.
So use a tool like Criterion to generate your test sequence values, but do
test these in the real instrument, and these also need to be tested with the
same hardware electrical signals actually present. |
|
![](templates/subSilver/images/spacer.gif) |
jeremiah
Joined: 20 Jul 2010 Posts: 1328
|
|
Posted: Thu Mar 10, 2022 12:18 pm |
|
|
We tend to distinguish between UNIT test and DVT (Design Verification Test). UNIT test is purely for testing code "paths", so you setup multiple sets of inputs in an attempt to verify every path in the code. For DVT, we hook up to actual signals and such to verify that the code design works as intended, treating the system as a black box and stressing all the inputs physically.
In our world UNIT test doesn't require an emulator. You generally create functions that call other functions/blocks of code with various hard coded input sets (some are generated by tools if we need random input sets) and each output is compared to an expected output of the function / block of code. |
|
![](templates/subSilver/images/spacer.gif) |
Ttelmah
Joined: 11 Mar 2010 Posts: 19358
|
|
Posted: Thu Mar 10, 2022 1:07 pm |
|
|
Absolutely.
Test things like how the maths behaves with an overflow, and that the
case statements do go where they are meant to go. |
|
![](templates/subSilver/images/spacer.gif) |
zaz
Joined: 18 Dec 2019 Posts: 2
|
|
Posted: Sun Mar 13, 2022 3:21 am |
|
|
What about Unity framework (for embedded software) for testing FW code. |
|
![](templates/subSilver/images/spacer.gif) |
Ttelmah
Joined: 11 Mar 2010 Posts: 19358
|
|
Posted: Mon Mar 14, 2022 10:34 am |
|
|
Unity is designed very much for .net languages. I don't know how easy it is
to adapt for languages. |
|
![](templates/subSilver/images/spacer.gif) |
enovak
Joined: 24 Nov 2021 Posts: 5
|
|
![](templates/subSilver/images/spacer.gif) |
Ttelmah
Joined: 11 Mar 2010 Posts: 19358
|
|
Posted: Wed Mar 16, 2022 7:55 am |
|
|
Well done.
As Jeremiah says, you then have to test the actual hardware operation. |
|
![](templates/subSilver/images/spacer.gif) |
bryant@balancebitsconsult
Joined: 21 Nov 2023 Posts: 38
|
|
Posted: Tue Mar 05, 2024 7:03 pm |
|
|
Here is an excellent list of unit test frameworks, for the purposes of testing that the code is accurate and returns the proper results.
https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks
However, for hardware testing nothing beats running the test on true hardware platforms, and this is specifically distinguished from unit tests. |
|
![](templates/subSilver/images/spacer.gif) |
Ttelmah
Joined: 11 Mar 2010 Posts: 19358
|
|
Posted: Wed Mar 06, 2024 5:03 am |
|
|
and for software testing as well...
Unit testing, is great for systems and code that is completely deterministic,
but will not catch faults that are not in the rules used. Unfortunately many
of the faults met inside microcontrollers fall into this category.
They are brilliant for proving that you are not doing something silly in
the handling of maths ranges, but will probably catch less than a quarter
of the real faults that can happen.... ![Sad](images/smiles/icon_sad.gif) |
|
![](templates/subSilver/images/spacer.gif) |
bryant@balancebitsconsult
Joined: 21 Nov 2023 Posts: 38
|
|
Posted: Wed Mar 06, 2024 11:50 am |
|
|
Ttelmah wrote: | and for software testing as well...
Unit testing, is great for systems and code that is completely deterministic,
but will not catch faults that are not in the rules used. Unfortunately many
of the faults met inside microcontrollers fall into this category.
They are brilliant for proving that you are not doing something silly in
the handling of maths ranges, but will probably catch less than a quarter
of the real faults that can happen.... ![Sad](images/smiles/icon_sad.gif) |
While true, splitting the problems into "unit tests caught this" and "integration with hardware caught this" make those problems easier to solve since the problem domains of each become more narrow (fewer things to research and implicate while troubleshooting). Using a unit testing framework therefore accelerates the hardware integration portions, and also encourages good habits for code structure so that issues CAN be seen.
Anyhow, I'm off topic now. Just wanted to share my 2 cents there on the topic of unit tests. I started on this thread examining what unit testing frameworks WOULD be easiest to integrate with the basic CCS toolchain - I haven't found a good example yet. |
|
![](templates/subSilver/images/spacer.gif) |
temtronic
Joined: 01 Jul 2010 Posts: 9168 Location: Greensville,Ontario
|
|
Posted: Wed Mar 06, 2024 6:11 pm |
|
|
hmm.. 'unit test frameworks' sounds about as reliable and trustworthy as every 'simulation' program I've ever seen. |
|
![](templates/subSilver/images/spacer.gif) |
bryant@balancebitsconsult
Joined: 21 Nov 2023 Posts: 38
|
|
Posted: Wed Mar 06, 2024 6:16 pm |
|
|
unit tests are at the crux of advanced software development for large systems.
In particular, test driven development (TDD) is something you may find useful to develop systems faster:
https://en.wikipedia.org/wiki/Test-driven_development
^ details of the benefits and outcomes you can expect when TDD is done properly. Of course, there are lots of ways to do it wrong and not see those benefits. ![Smile](images/smiles/icon_smile.gif) |
|
![](templates/subSilver/images/spacer.gif) |
|