How to properly send bulk packets from vendor device greater than 64 bytes

[second-string]

I'm trying to send 500-600 bytes from my custom vendor device back to the host machine, but I can't figure out how to properly chunk it out of the output endpoint. Things I've tried:

1. Calling tud_vendor_write(buf, 64) repeatedly, advancing the buf pointer each them. When full packet sent, call tud_vendor_write_flush()
2. Calling tud_vendor_write(buf, 64) and then tud_vendor_write_flush() for every 64 byte chunk until entire packet is done
3. Calling tud_vendor_write(buf, 500) and then tud_vendor_write_flush().
4. Calling while (tud_vendor_write_available() == 0) {} before calling tud_vendor_write(buf, 64) and tud_vendor_write_flush(). But this spins forever and never exits that while loop because it always returns 0

None of these are working properly when viewed with Wireshark on the receiving host. Am I supposed to increase the size of the TX FIFO by overriding the default CFG_TUD_VENDOR_TX_BUFSIZE of 64 to a higher value? Or am I somehow chunking this data out incorrectly? I feel like I'm missing something simple.

Also probably worth noting that all of my calls to tud_vendor_write and tud_vendor_write_flush always return zero, even though this behaviour is all working as expected for packets < 64 bytes length.

[second-string]

Okay I've partially answered my question. In my application, I'm using a vendor device to emulate and existing hardware device. This device prefixes every message with 2 status bytes, so each data packet is 62 bytes.

In all my examples, I was sending this 64 byte packet with 2 prefix and 62 data bytes. TinyUSB then was automatically sending a ZLP, because that's the typical behavior when trying to flush the full buffer. This was breaking how a lot of software parsed incoming data from these devices, making it seem like the firmware was the issue. Writing 1 less data byte and then calling the flush function made this work.

I also discovered how to properly increase the buffer size itself. Previously, when I increased the TX buffer size for tinyusb (that's used for the FIFO, not the hardware packet size of USB high speed obviously), I must have not been flushing properly, because in wireshark I was seeing these really slow, big chunks come in. Now, I can write all 500-600 bytes into tinyusb in one write, then flush it one time, which sends out X 64-byte high speed packets seen in wireshark.

I have everything working with most parsing software, the most basic being screen reading it serially. There are still a few pieces of software that are reading values incorrectly if their parsing is started after tinyusb has started writing / flushing the buffer. They work fine if the transmitting is started after the host opens the connection. I think there's still something small incorrect with the buffering and flushing, but I can't test further until a week or so.

[kuba2k2]

Hi @second-string
I've seen your other post in TinyUSB's discussions, and it seems like we're trying to implement/emulate the same vendor device 😄
Can you share how you were able to solve the bulk packet size issues?

In all of my attempts, the data that comes back (viewed in Wireshark) is completely broken, like the packets are too long/too short, or the data is mixed up between packets (e.g. the 2 status bytes land in the middle of a packet.

My general flow is:

1. Receive data from a FIFO (not TinyUSB's FIFO, but one internal to the emulated device) - the data is written to the FIFO by other parts of the program.
2. Write it to the USB, preceding each 62-byte chunk with a 2-byte status.
3. Flush the TX buffer (if enabled).

As such, I have tried two different approaches, with TinyUSB's TX buffer disabled (set to 0) and enabled (set to 4096):

1. Buffer disabled - build a 4096-byte package with the status bytes and actual data correctly placed, then call tud_vendor_write() to send the entire packet. This works for small packets (under 64 bytes, exactly 64 bytes won't work). Larger packets arrive with a 4-5 second delay, as a 4096 byte chunk of data.
2. Buffer enabled - send each 64 bytes (2+62) with tud_vendor_write(). After sending all small packets (which adds up to 4096 bytes total), call tud_vendor_write_flush(). This works for sending larger amounts of data (full 4k), but doesn't always work when sending small packets. The _flush() call simply doesn't work properly, it doesn't transmit the data everytime when called.

Consider the following code, being run in a loop with some delay in between iterations (buffer size is 4096):

		uint8_t packet[64];
		packet[0] = 0x31;
		packet[1] = 0x60;
		tud_vendor_write(packet, 2);
		tud_vendor_write_flush();

which yields the following result:

The small packets are sometimes sent separately, but then are concatenated into larger, 4096 byte packets. And that indeed breaks pretty much all software that is supposed to read these packets.

---

Have you found a way to make this work properly? It seems like sending the data without a buffer could be a way to go, but I just couldn't figure out the right way to do it yet.
On the other hand, I'm not sure how to make the flush method work when using a buffer. Maybe there's a way to force sending a ZLP which might "split" the packets logically?

Thanks in advance!

[second-string]

Very busy and on the road for the next week or so, but will circle back to this when I have time!

[second-string]

I was running into a couple different issues. The first was trying to send packets exactly the size of the buffer, 64 bytes. That's what I was referring to here:

In all my examples, I was sending this 64 byte packet with 2 prefix and 62 data bytes. TinyUSB then was automatically sending a ZLP, because that's the typical behavior when trying to flush the full buffer. This was breaking how a lot of software parsed incoming data from these devices, making it seem like the firmware was the issue. Writing 1 less data byte and then calling the flush function made this work.

I had two fixes for this - either send less than 64 total bytes at a time and manually flush, or increase the buffer size and manually flush. I got the first working first, then moved to the second because it's much more efficient for my ~500 byte packet size.

On the other hand, I'm not sure how to make the flush method work when using a buffer. Maybe there's a way to force sending a ZLP which might "split" the packets logically?

tinyusb automatically sends the ZLP when you send a buffer into the TX FIFO that equals the exact length of the hardware USB buffer, 64. I don't know enough about the USB full speed spec to understand the specifics around this, but it seemed like that was causing issues for the software parsing my packets.

I would also recommend looking at the software side for potential issues as well. I set up a simple python script that was pulling in data from my tinyusb device and it showed that a lot of the stuff that I had been doing was working fine, it was just inefficient or bad coding in the software applications I was testing that was causing bottlenecks. You mentioned that the 4096 bytes is taking 4-5 seconds to transmit - i was also seeing weirdly long big chunk delays of 1-3 seconds where the data would pile up and then get transferred all at once, and it was just due to the software. The python worked just fine.

Lastly, are you adding in the status bytes in the first two bytes of every 64 byte chunk that you're sending, even if it's all in one 4096 buffer? The vendor driver is only reading 64 bytes at a time and it needs that prefix for every chunk. This is the loop I have to build my tx packet before calling tud_vendor_write

    uint16_t remaining_data  = data_len;
    uint16_t tx_data_len     = 0;
    uint8_t *moving_dest_ptr = tx_packet;
    uint8_t *moving_src_ptr  = data;
    uint8_t  num_chunks      = 0;
    while (remaining_data) {
        *moving_dest_ptr++ = 0x31;
        *moving_dest_ptr++ = 0x60;
        tx_data_len        = MIN(remaining_data, 62);
        memcpy(moving_dest_ptr, moving_src_ptr, tx_data_len);
        moving_dest_ptr += tx_data_len;
        moving_src_ptr += tx_data_len;
        remaining_data -= tx_data_len;
        num_chunks++;
    }

[kuba2k2]

It's been a while since I worked on this emulation project... in the end I managed to solve it with this code:

	if ((tx_len || timed_out) && tud_vendor_n_write_available(0)) {
		uint8_t packet[64];
		// write line status as first 2 bytes of every packet
		memcpy(packet, &ft->modem_stat, 2);
		uint32_t read_len;
		if (tx_len > 62)
			// read 62 bytes if there's more data in the TX FIFO
			read_len = tu_fifo_read_n(&ft->tx_ff, packet + 2, 62);
		else if (tx_len == 62)
			// avoid reading exactly 62 bytes, as that stalls the transfer for some reason
			read_len = tu_fifo_read_n(&ft->tx_ff, packet + 2, 61);
		else // tx_len < 62
			// just read all the data
			read_len = tu_fifo_read_n(&ft->tx_ff, packet + 2, tx_len);
		// decrement the TX length if data was read, otherwise assume the FIFO is empty
		if (read_len != 0 && tx_len >= read_len)
			tx_len -= read_len;
		else
			tx_len = 0;
		// write to the endpoint
		tud_vendor_write(packet, 2 + read_len);
	}

This runs in a loop, right after tud_task(). I don't remember the specific issues I had anymore, but this solved most (all?) of them, so the emulated device behaved just like the real hardware.

To answer the question, yes, I remembered to add the status bytes every other 64 bytes.