CCS C Software and Maintenance Offers
FAQFAQ   FAQForum Help   FAQOfficial CCS Support   SearchSearch  RegisterRegister 

ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

CCS does not monitor this forum on a regular basis.

Please do not post bug reports on this forum. Send them to support@ccsinfo.com

Unit Test Framework
Goto page 1, 2  Next
 
Post new topic   Reply to topic    CCS Forum Index -> General CCS C Discussion
View previous topic :: View next topic  
Author Message
enovak



Joined: 24 Nov 2021
Posts: 5

View user's profile Send private message

Unit Test Framework
PostPosted: Thu Dec 09, 2021 9:25 am     Reply with quote

I've been researching implementing some automated unit test in my project. I know that there are a number of different frameworks which are available to use. I was curious if anyone has implemented anything using the CCS compiler. Any input on the best framework to use? Are you executing your tests on target or simulating them? Thanks for the input.
zaz



Joined: 18 Dec 2019
Posts: 2

View user's profile Send private message

PostPosted: Thu Mar 10, 2022 3:19 am     Reply with quote

I am also looking for unit test framework for CCS project.
temtronic



Joined: 01 Jul 2010
Posts: 9081
Location: Greensville,Ontario

View user's profile Send private message

PostPosted: Thu Mar 10, 2022 6:21 am     Reply with quote

I can't see any UTF or another 'simulator' actually being 100% accurate with using a simple PIC like the 16C84, let alone a modern version, say a 46K22.
The problem I see is that the UTF programmer has no hands on, real world experience with a PIC ? Just HOW does he emulate random cross talk that feeds into the 2nd UART and corrupts the data stream ? Or the effects of a nearby lightning bolt ?
Ttelmah



Joined: 11 Mar 2010
Posts: 19195

View user's profile Send private message

PostPosted: Thu Mar 10, 2022 6:40 am     Reply with quote

Dead right.
I have used Criterion to make test values to prove functionality of some
code in the past, but the problem is that instrumentation code is far more
likely to have problems from things like EMP, than from general 'code errors'.
I have yet to see any simulator that I would rate as being more than 99%
accurate, and all will miss 'oddities' with particular chips, that the real
instrument has to handle.
So use a tool like Criterion to generate your test sequence values, but do
test these in the real instrument, and these also need to be tested with the
same hardware electrical signals actually present.
jeremiah



Joined: 20 Jul 2010
Posts: 1314

View user's profile Send private message

PostPosted: Thu Mar 10, 2022 12:18 pm     Reply with quote

We tend to distinguish between UNIT test and DVT (Design Verification Test). UNIT test is purely for testing code "paths", so you setup multiple sets of inputs in an attempt to verify every path in the code. For DVT, we hook up to actual signals and such to verify that the code design works as intended, treating the system as a black box and stressing all the inputs physically.

In our world UNIT test doesn't require an emulator. You generally create functions that call other functions/blocks of code with various hard coded input sets (some are generated by tools if we need random input sets) and each output is compared to an expected output of the function / block of code.
Ttelmah



Joined: 11 Mar 2010
Posts: 19195

View user's profile Send private message

PostPosted: Thu Mar 10, 2022 1:07 pm     Reply with quote

Absolutely.
Test things like how the maths behaves with an overflow, and that the
case statements do go where they are meant to go.
zaz



Joined: 18 Dec 2019
Posts: 2

View user's profile Send private message

PostPosted: Sun Mar 13, 2022 3:21 am     Reply with quote

What about Unity framework (for embedded software) for testing FW code.
Ttelmah



Joined: 11 Mar 2010
Posts: 19195

View user's profile Send private message

PostPosted: Mon Mar 14, 2022 10:34 am     Reply with quote

Unity is designed very much for .net languages. I don't know how easy it is
to adapt for languages.
enovak



Joined: 24 Nov 2021
Posts: 5

View user's profile Send private message

PostPosted: Wed Mar 16, 2022 6:44 am     Reply with quote

@zaz I did end up using Unity and getting it to work with the Microchip simulator. I did have to mock out some of the HW interactions. Using mocks is going to be key to helping to ensure that all of the code paths are tested and confirmed to be operating correctly.

Here are a couple of links that I referenced in the work that I was doing:
* http://www.throwtheswitch.org/unity
* https://spin.atomicobject.com/2012/10/22/getting-started-with-tdd-for-microchips-pics/
Ttelmah



Joined: 11 Mar 2010
Posts: 19195

View user's profile Send private message

PostPosted: Wed Mar 16, 2022 7:55 am     Reply with quote

Well done.
As Jeremiah says, you then have to test the actual hardware operation.
bryant@balancebitsconsult



Joined: 21 Nov 2023
Posts: 21

View user's profile Send private message

PostPosted: Tue Mar 05, 2024 7:03 pm     Reply with quote

Here is an excellent list of unit test frameworks, for the purposes of testing that the code is accurate and returns the proper results.

https://en.wikipedia.org/wiki/List_of_unit_testing_frameworks

However, for hardware testing nothing beats running the test on true hardware platforms, and this is specifically distinguished from unit tests.
Ttelmah



Joined: 11 Mar 2010
Posts: 19195

View user's profile Send private message

PostPosted: Wed Mar 06, 2024 5:03 am     Reply with quote

and for software testing as well...

Unit testing, is great for systems and code that is completely deterministic,
but will not catch faults that are not in the rules used. Unfortunately many
of the faults met inside microcontrollers fall into this category.
They are brilliant for proving that you are not doing something silly in
the handling of maths ranges, but will probably catch less than a quarter
of the real faults that can happen.... Sad
bryant@balancebitsconsult



Joined: 21 Nov 2023
Posts: 21

View user's profile Send private message

PostPosted: Wed Mar 06, 2024 11:50 am     Reply with quote

Ttelmah wrote:
and for software testing as well...

Unit testing, is great for systems and code that is completely deterministic,
but will not catch faults that are not in the rules used. Unfortunately many
of the faults met inside microcontrollers fall into this category.
They are brilliant for proving that you are not doing something silly in
the handling of maths ranges, but will probably catch less than a quarter
of the real faults that can happen.... Sad



While true, splitting the problems into "unit tests caught this" and "integration with hardware caught this" make those problems easier to solve since the problem domains of each become more narrow (fewer things to research and implicate while troubleshooting). Using a unit testing framework therefore accelerates the hardware integration portions, and also encourages good habits for code structure so that issues CAN be seen.

Anyhow, I'm off topic now. Just wanted to share my 2 cents there on the topic of unit tests. I started on this thread examining what unit testing frameworks WOULD be easiest to integrate with the basic CCS toolchain - I haven't found a good example yet.
temtronic



Joined: 01 Jul 2010
Posts: 9081
Location: Greensville,Ontario

View user's profile Send private message

PostPosted: Wed Mar 06, 2024 6:11 pm     Reply with quote

hmm.. 'unit test frameworks' sounds about as reliable and trustworthy as every 'simulation' program I've ever seen.
bryant@balancebitsconsult



Joined: 21 Nov 2023
Posts: 21

View user's profile Send private message

PostPosted: Wed Mar 06, 2024 6:16 pm     Reply with quote

unit tests are at the crux of advanced software development for large systems.

In particular, test driven development (TDD) is something you may find useful to develop systems faster:
https://en.wikipedia.org/wiki/Test-driven_development

^ details of the benefits and outcomes you can expect when TDD is done properly. Of course, there are lots of ways to do it wrong and not see those benefits. Smile
Display posts from previous:   
Post new topic   Reply to topic    CCS Forum Index -> General CCS C Discussion All times are GMT - 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group