How Can I Resolve Buffer Overflow for Asynchronous CAN Messages?

A Question from the Customer:

I have a question about communicating with an external CAN device. I'm working on a PC application to transfer data over the Komodo CAN Duo Interface. The bus is configured to run at 1 MHz and I’m starting this project with 16KB of data. When that is successful, the data blocks will be increased to megabytes of data.

With the current architecture, data is split into 32-byte "packets" plus five bytes of header information. This packet structure is then split into 8-byte CAN packets for transfer via the Komodo interface. Each of the 37-byte packets is acknowledged with a 2-byte ACK message from the receiver. The five 8-byte CAN packets are not ACK’d individually. I am using the Komodo Software API.

Here’s a summary of what I’m doing:

  1. Call the km_can_async_submit() function 5 times with the 37 bytes of data split into 4 8-byte messages and 1 5-byte message.
  2. Call km_can_read() configured with a 10 second timeout and wait for a successful 2-byte acknowledge response before continuing with the next 37-byte transmission.
  3. Repeat this process until the entire 16 KB of data has been transferred.

The known issues and fixes:

I can send the first 12 37-byte packets successfully and receive an acknowledgement. However:

  1. No acknowledgement is received for the 13th
  2. I’ve worked with the Komodo timeout value as well as the latency. Lowering the latency does make the process faster before failing, but the failures still occur.
  3. I've also tried multiple calls to km_can_read() thinking the missing ACK message got buried under some empty messages, but even after multiple calls to km_can_read() the responses are always 0.

The “work around” that I am using:

  • I can transfer data by resetting the Komodo interface after every 11 packets by calling km_close() and KomodoApi.km_open().

The improvements I’m looking for:

  • It works – but is there a way to do with without closing and re-opening the transmission? Is there a buffer that’s not being cleared unless the Komodo interface is re-initialized?

Response from Technical Support:

Thanks for your question! As implied in your statement, the issues you observed are due to buffer overflow. The solution we have for you is relatively simple. To better understand how the solution works, we will first go over the mechanics of latency settings, asynchronous messages, and what affects the data buffer.

The Effects of Latency Settings

Setting a small latency can increase the responsiveness of the read function. Please note, there is a fixed cost to processing each individual buffer that is independent of buffer size. Here are the trade-offs to consider:

  • Setting a small latency value increases the overhead per byte that is buffered.
  • Setting large latency value decreases that overhead, but it increases the amount of time that the library must wait for each buffer to fill before the library can process its contents.

How the Latency Function Works

The Komodo km_latency function sets the capture latency to the specified number of milliseconds. For example, if latency_ms is set to “1”, then the latency is 1ms.

The capture latency effectively splits up the total amount of buffering into smaller individual buffers (sub-buffers). After one of these individual buffers is filled, the read function returns.

  • To fulfill shorter latency requirements, these individual buffers are set to a smaller size.
  • If a larger latency is needed, then the individual buffers are set to a larger size.

In other words, if the latency_ms parameter value in the km_latency function is small, then the sub-buffers size is small. If the latency_ms parameter value in the km_latency function is big, then the sub-buffers size is big.

Latency vs. Timeout

The latency setting is distinctly different from the timeout setting. Latency should be set to a value that is less than the timeout.

Buffering Asynchronous Messages

There is buffering within the Komodo DLL, on a per-device basis, to help capture asynchronous messages. Here is a case of a Komodo interface receiving CAN messages asynchronously:

  • If the application calls the function to change the state of a GPIO while unprocessed asynchronous messages are pending, the Komodo interface will modify the GPIO pin, but also save any pending CAN messages internally.
  • The pending messages will be held until the appropriate API function is called.

Active and Passive CAN Modes

The Komodo interface can be configured as an active CAN node, or as a passive CAN monitor. A CAN channel can receive messages asynchronously with respect to the host PC software. Between calls to the Komodo API, these messages must be buffered somewhere in memory. This buffering is accomplished by the operating systems of the PC host. The buffer is limited in size; when the buffer is full, bytes will be dropped.

Buffer Overflow and Asynchronous Messages

An overflow occurs when the Komodo interface receives asynchronous messages faster than the rate at which they are processed: the receive link becomes "saturated". This condition can also affect synchronous communication with the Komodo interface.

How to Alleviate Buffer Saturation

There are two ways to relieve the receive saturation problem:

  • Reduce the amount of traffic that is sent by all CAN nodes between calls to the Komodo API. If you use this method, you will need to reconfigure the offending CAN device(s).
  • Poll the CAN channel to collect pending messages more frequently.

API Solution

Based on your request and setup, we recommend using the API command km_can_write with your script. This function acts as a wrapper for the asynchronous submit and collect functions.

km_can_async_submit(komodo, channel, flags,
packet, num_bytes, data);
km_can_async_collect(komodo, KM_TIMEOUT_INFINITE,
arbitration_count);

How this works:

  • The CAN packet is submitted asynchronously. km_can_async_collect is called with KM_TIMEOUT_INFINITE to continue blocking until a response is received.
  • A KM_CAN_ASYNC_PENDING error is returned if there are any uncollected asynchronously submitted packets.

 Note: Packets that are submitted with km_can_async_submit should always be collected using km_can_async_collect.

Additional resources that you may find helpful include the following:

We hope this answers your question. If you need more information, you can contact us and we’ll go over your requirements. You can also request a demo that applies to your application.

Request a Demo