 |
 |
| View previous topic :: View next topic |
| Author |
Message |
Kamal
Joined: 11 Feb 2006
Posts: 12
|
| Towards a reliable Serial comm. lib over long distances |
 Posted: Sat Feb 11, 2006 9:32 pm |
 |
|
Hi
I want to start a project to implement a reliable Serial communications library meant to exchange data mainly between a PC and a PIC (877A) over a RS422 link over long distances.
Hence, I need some code to serve as a basic foundation toards building a complete library (which I aim to write using CCS C) for :
1. Asynchronous serial communication
2. Transfer a block of data (say a byte) a time and verify it's intergrity. If the data has been corrupted, the data should be sent anew
These are my basic requirments. If somebody has done something like this before, sharing ideas would be great !
Please offer your suggestions and ideas. |
|
 |
Humberto
Joined: 08 Sep 2003
Posts: 1215
Location: Buenos Aires, La Reina del Plata
|
|
| Posted: Sun Feb 12, 2006 12:24 pm |
|
|
Hi Kamal,
You are new in this forum. It is improbable that somebody will just start writting code
for you after your inquiry. We can help you but the first movement is yours.
You would start reading and searching in this forum how another people had solved
similar problems, then write your own code and post it.
Surelly you will get a lot of help in this way.
Humberto |
|
|
jds-pic
Joined: 17 Sep 2003
Posts: 205
|
| Re: Towards a reliable Serial comm. lib over long distances |
| Posted: Sun Feb 12, 2006 12:44 pm |
|
|
| Kamal wrote: | Hi
I want to start a project to implement a reliable Serial communications library meant to exchange data mainly between a PC and a PIC (877A) over a RS422 link over long distances.
Hence, I need some code to serve as a basic foundation toards building a complete library (which I aim to write using CCS C) for :
1. Asynchronous serial communication
2. Transfer a block of data (say a byte) a time and verify it's intergrity. If the data has been corrupted, the data should be sent anew
These are my basic requirments. If somebody has done something like this before, sharing ideas would be great !
Please offer your suggestions and ideas. |
i've done this lots of times over long distances using RS422 (pt-to-pt diff erential RS232) and RS485 (multi-drop differential). i would first recommend reading quite a bit about RS422, RS485, ground loops, and galvanic isolation. there are some great how-to's on the following website;
http://www.robustdc.com/
see the tech resources section, under appnotes:
http://www.robustdc.com/techResources-appnotes.htm?a=8
you absolutely need to get the physical layer right! if the line is noisy and the received data is suspect, no amount of software is going to make it work right. make sure that you get the physical layer sorted out before starting your software design. the pipe has work.
as for a protocol, you have not described how your endpoints interact. is it a master/slave relationship? does the slave generate asynchronous events or can you just set it up for the master to poll the slave? i'll assume that the PC is the master and the PIC is the slave.
the absolute easiest way to do this is to have a fixed length packet. make it long enough to carry your largest data payload, and short enough to protect with a reasonable checksum.
example, an 8 byte packet:
byte 1: target node
byte 2: source node
byte 3: opcode (aka command byte)
byte 4: payload byte 1
byte 5: payload byte 2
byte 6: payload byte 3
byte 7: payload byte 4
byte 8: checksum
in a point to point network you may not need the first two bytes; however, keeping them there allows you to easily grow your network into a multi-node arrangement. this is how you write a library once, by making it extensible. you'll note that the addressing scheme above also supports broadcasting, using a target address of FF.
a basic transaction looks like this:
PC sends
01 00 03 00 00 00 00 ck
"hey 01, this is 00, please send me the temperature (opcode 03), payload is all don't care, checksum"
PIC replies
00 01 03 23 00 00 00 ck
"hey 00, this is 01, here are the results of your opcode 03 query, data1 is 23 -- rest are don't care, checksum"
there are some error conditions you will have to handle as well:
PC sends
01 00 52 05 01 00 00 ck
"hey 01, this is 00, please set switch (opcode 52) data0 to the value in data1, data0 is 5 -- data1 is 1 -- rest are don't care, checksum"
PIC replies
00 01 FF 00 00 00 00 ck
"hey 00, this is 01, FF=illegal opcode received or payload parameter out of range, payload is all don't care, checksum"
using a fixed length packet has some advantages and some drawbacks. one advantage is that it simplifies the capture state machine. one drawback is that it fixes the latency to the size of the packet. so you have to trade off here.
for a checksum routine i would employ either Xor'ing of the bytes (simple and fast) or better yet use the dallas onewire CRC routine. the latter is very robust and easy to implement. it is perfectly suited for 8 byte packets, by the way. i can point you to the dallas CRC code in CCS C if you need.
hope that gets you started off on the right foot,
jds-pic |
|
|
Kamal
Joined: 11 Feb 2006
Posts: 12
|
|
| Posted: Sun Feb 12, 2006 6:09 pm |
|
|
Thanks for the great links. I DONOT expect people to write code for me, but rather point me to code they have already written or seen..
The PIC will interrupt the PC when is has some data to send. The PC NEVER starts th data transaction.
I want to use a completely asynchronous serial link with error detection and retransmission if error(s) occur.
Also, as I am thinking of a Point-to-point protocol, with a PIC and a PC on either end, I think that specifiying the target and source nodes are not necessary (but if sufficient no. people start using the library, we can always add it)
I am interested in the dallas CRC code in CCS C - is it also available in the demo version ? Then I will download it and start working away.
Thanks again for the great help. |
|
|
kender
Joined: 09 Aug 2004
Posts: 768
Location: Silicon Valley
|
| Re: Towards a reliable Serial comm. lib over long distances |
| Posted: Sun Feb 12, 2006 6:49 pm |
|
|
I was waiting for a conversation on the subject of reusable protocol to start. I think it will be very useful if we define a communication protocol spec/guideline, which can work RS232, RS485, CAN and I2C physical layers. There were a lot requests for a protocol spec in this forum (we even clashed with Humberto over it once). Let's define a spec - the code will eventually follow.
| jds-pic wrote: |
example, an 8 byte packet:
byte 1: target node
byte 2: source node
byte 3: opcode (aka command byte)
byte 4: payload byte 1
byte 5: payload byte 2
byte 6: payload byte 3
byte 7: payload byte 4
byte 8: checksum
|
Here is a variable-length version:
| Code: |
byte 1: target node
byte 2: source node
byte 3: opcode (aka command byte)
byte 4: source's protocol version (let's make it more a less backwards-compatible)
byte 5: payload length
byte 6: payload byte 1
byte 7: payload byte 2
byte 8: payload byte 3
byte 9: payload byte 4
byte 10: checksum
|
P.S. Darren, since the title is generic enough, it would be nice if we could make this thread a sticky.
Last edited by kender on Mon Feb 13, 2006 1:17 am; edited 1 time in total |
|
|
Kamal
Joined: 11 Feb 2006
Posts: 12
|
|
| Posted: Sun Feb 12, 2006 8:11 pm |
|
|
Excellent !
A reusable library will be usable by everybody. I just can't tell you how many PIC programmers scramble here and there and spam forums just to do some basic, reliable serial communications !
The protocol definition seems generic enough, altough I think that if we are going to use handshaking (initally software - say XON/XOFF) then we also need to number the ACKs ? This will help us deal with delayed ACKs and lost packets - what do you think ?
I also will like to push CRC8 as a means to verify the intergrity of each packet.
I glossed over the RS422 links you sent me, and I may have a good hardware/physical layer design soon. But maybe CCS Forum is not the place for that ?
I also saw the Modbus Onewire library and they seem to have pretty nice code. So how do I get started with writing this library ? I am willing to take the initial brunt of it, but I need good guidance. |
|
|
PCM programmer
Joined: 06 Sep 2003
Posts: 21708
|
|
| Posted: Sun Feb 12, 2006 9:26 pm |
|
|
Or instead of "sticky", you could request a new forum, called "Projects".
Just an idea. |
|
|
jds-pic
Joined: 17 Sep 2003
Posts: 205
|
|
| Posted: Sun Feb 12, 2006 9:46 pm |
|
|
| Kamal wrote: |
The PIC will interrupt the PC when is has some data to send. The PC NEVER starts th data transaction. |
programmatically, it will be easier to do this the other way around. have the PC poll the PIC on a fixed interval. the interval can be very short, even as short as say just one byte time (at 9600bps that's 1ms). the reason i suggest this is that it makes writing the Window's/Linux application easier.
| Kamal wrote: |
I want to use a completely asynchronous serial link with error detection and retransmission if error(s) occur. |
if you expect that one day you will have more than two entities on the network (for example, in an RS485 multi-drop arrangement), then you have to design in some order from the get go. you can't have a free for all, since you will have to add hardware and a lot of complexity to detect collisions and so forth. see the "cheapernet" Ethernet over coax implementation for details on CSMA/CD, one possible approach. the other approach is to make the communications deterministic, usually by having one guy in charge (the master PC) and the slaves are polled (the PICs). there can be other means of having the PICs interrupt the PC for time-sensitive issues.
i implore you to work out a deterministic solution, otherwise you will have challenging systems design and programming tasks ahead of you. the KISS principle rules here. for example, you should consider a few problems such as:
PC polls PIC for data; data is not yet ready, will take PIC about 3 seconds to gather and filter raw info before it can return the data. when does the PC declare query timeout? 3 seconds is a long time to sit waiting on one specific client PIC, when others may need attention.
PC polls PIC for data; data is not yet ready, will take PIC about 3 seconds to gather and filter raw info before it can return the data. can the PC poll other PICs during this interval? or can there only be one "open" transaction on the bus?
| Kamal wrote: |
Also, as I am thinking of a Point-to-point protocol, with a PIC and a PC on either end, I think that specifiying the target and source nodes are not necessary (but if sufficient no. people start using the library, we can always add it) |
i would put it in from the start. folks can always ignore the target/source bytes for pt-pt applications.
| Kamal wrote: | | I am interested in the dallas CRC code in CCS C - is it also available in the demo version ? Then I will download it and start working away. |
http://www.ccsinfo.com/forum/viewtopic.php?t=19520
see the function
int onewire_crc(int oldcrc, int newbyte)
note: you can also do it via a lookup table if you have the ROM space.
also, refer to the top of page 3 in the referenced Dallas Semi PDF document for the error detection coverage of this particular CRC implementation when protecting an 8 byte packet.
jds-pic |
|
|
jds-pic
Joined: 17 Sep 2003
Posts: 205
|
|
| Posted: Sun Feb 12, 2006 9:52 pm |
|
|
| Kamal wrote: | | So how do I get started with writing this library ? I am willing to take the initial brunt of it, but I need good guidance. |
i will post a "starting place" protocol spec document tomorrow. i did it for a project which was subsequently cancelled and we might as well put it to good use.
jds-pic |
|
|
kender
Joined: 09 Aug 2004
Posts: 768
Location: Silicon Valley
|
| Re: Towards a reliable Serial comm. lib over long distances |
| Posted: Mon Feb 13, 2006 1:36 am |
|
|
| jds-pic wrote: |
example, an 8 byte packet:
byte 1: target node
byte 2: source node
byte 3: opcode (aka command byte)
byte 4: payload byte 1
byte 5: payload byte 2
byte 6: payload byte 3
byte 7: payload byte 4
byte 8: checksum
|
Hey JSD,
I'm probably missing a piece here... I assume this was a binary (as opposed to ASCII) protocol. How did you signal the beginning or the end of the packet? Did you send a break? Did you have additional handshake lines? Was the protocol strictly synchronous, and the PIC would not talk unless it was queried?
I have some code and specs for a ASCII protocol, where all data was sent as hex-ASCII. E.g. the number 254 would be sent as a pair of characters "FE" (2 bytes). Packets had variable length. '$' character was a start marker. As an ASCII protocol it had a lower throughput, but on the up side was easy to code and debug. It also required minimal hardware - just RS-232 with TX and RX. |
|
|
Kamal
Joined: 11 Feb 2006
Posts: 12
|
| Let's KISS by starting simple |
| Posted: Mon Feb 13, 2006 4:47 am |
|
|
Hello mates
I am quite amazed with the amount of ideas coming up, but remember, it's <b><i>I</i></b> who is going to do/start the coding ;)
So currently, altough I will keep the framework ready for future expansion, I am only interested in <i>serial Point-to-point</i> transmission using RS422. Multipoint will be too complex for me to handle at the moment, as so will be testing the code as I don't have such a framework and testing/building so will take considerable time and money.
So I was thinking of :
1. Starting off by defining a RS422 based long (max 4000FT) distance protocol which can be used to provide reliable (but not necessarily fast) data transmission.
Regarding the RS422 based design, I would settle for a optoisolated (at least on one end) design with 4-core (or 2-core) shielded wire
2. As we will be basically using software handshaking to start with, I was thinking along the lines of <b>jds-pic</b> of using a ASCII based protocol as that's the ONLY way we can implement XON/XOFF. Ofcourse, I will be too glad to be wrong regarding this. But seriously, implementing a binary protocol without more wires seem difficult. I anybody has better ideas, <i>please</i> share.
3. Polling the PIC for data will make the PC side driver/code easier, but consider that the SAME PC might be connected to 4 or more such PICs. <i>Now can you estimate the performance of the PC because of polling overhead</i> ?
However, to get something useful to start with, I will happily incorporate polling. It's better to have something at hand than spend months debating what should be good design.
I am eager to start soon, and keep the creative juices flowing. <b>jds-pic</b> has been most helpful with his one wire library.
So let's go !
Last edited by Kamal on Mon Feb 13, 2006 5:14 am; edited 1 time in total |
|
|
kender
Joined: 09 Aug 2004
Posts: 768
Location: Silicon Valley
|
|
| Posted: Mon Feb 13, 2006 5:10 am |
|
|
| Kamal wrote: | | Polling the PIC for data will make the PC side driver/code easier,.. |
Many people say that polling is easier. Personally, I'm not convinced that this is true with modern PCs and modern programming languages that are event-based (such as VB or C++). Here’s my rationale… Case 1 - the code that does polling (or in other words – synchronous communication) has to have a state machine that covers both sending and receiving data. Case 2 - the code that does asynchronous communication has to have two separate state machines and their cumulative complexity is not greater then in the case 1. Two smaller state machines are easier to debug then one bigger one. EDIT: I have point-to-point communication in mind.
In addition, asynchronous communication works better in the fault/error conditions. It can signal the fault to the host without having to wait until it’s polled.
Last edited by kender on Mon Feb 13, 2006 3:39 pm; edited 1 time in total |
|
|
Kamal
Joined: 11 Feb 2006
Posts: 12
|
|
| Posted: Mon Feb 13, 2006 5:31 am |
|
|
My thoughts exactly, kender
Not only would event/interrupt based handling be easier, but it would be efficient too.. but anyways, that does NOT change our protocol too much, just that the PIC will have to notify the PC. But then can we do with just Tx/Rx ? Will not RTS etc be required ?
By the way, we should start with a good design soon and have a prototype in our hands within <i><b>this week</b></i>.
It's easy to go on talking what good design is, but the moment one start <i>implementing</i> the design, most of the "architects" slither away ;)
I want this library to stay. In all PIC forums I have seen so far, 98% of threads are RS232 sreial communication based.
I think it's high time we put together the collective intelligence of all the gurus and solve the problem once and for all.
Next time <i>anyone</i> asks a serial communications library, we know where to point them to ;)
Let's start soon. |
|
|
jds-pic
Joined: 17 Sep 2003
Posts: 205
|
|
| Posted: Mon Feb 13, 2006 12:40 pm |
|
|
| jds-pic wrote: | | Kamal wrote: | | So how do I get started with writing this library ? I am willing to take the initial brunt of it, but I need good guidance. |
i will post a "starting place" protocol spec document tomorrow. i did it for a project which was subsequently cancelled and we might as well put it to good use.
jds-pic |
http://losdos.dyndns.org:8080/public/pic-uC/cbus-protocol-spec-draft-rev1.pdf
jds-pic |
|
|
jds-pic
Joined: 17 Sep 2003
Posts: 205
|
|
| Posted: Mon Feb 13, 2006 12:49 pm |
|
|
| Kamal wrote: | Excellent !
The protocol definition seems generic enough, altough I think that if we are going to use handshaking (initally software - say XON/XOFF) then we also need to number the ACKs ? This will help us deal with delayed ACKs and lost packets - what do you think ? |
i think you may be overcomplicating things here. remember "KISS" -- it's easier to design, code, and debug. if you want to use XON/XOFF you have to escape any payload values that correspond with XON/XOFF. this in turn complicates packet construction, checksumming, and capture. really, for reasonable size packets (4 to 32 bytes) at reasonable speeds (1.2-57.6Kbps) there should be no need to use any kind of flow control, SW or HW. the PIC can manage this throughput just fine, and of course the PC can as well.
jds-pic |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ 
Search
Register
Profile 
Log in to check your private messages 
Log in
