Error while placing tonies with radio stream

Transistor · September 15, 2025, 7:17pm

Hi all,

Since a few weeks I am facing a strange behavior with my Tonie box / TeddyCloud.
I have 10 custom Tonies assigned to a radio stream. When starting up the box and placing one of them I all the time get the “owl => check internet connection” and box is blinking red. Afterwards I have to restart it, and do a freshness check. Afterwards the radio stream is playing as expected. Placing a different custom Tonie with a radio stream assigned has the same behavior.

Any Idea what can cause this?

Running Teddy cloud on Raspberry Pi4 in docker with portainer. My box is a ESP32 and I have around 75 original Tonies + 10 custom once with radio streams assigned. I somehow have the doubt that the box is running out of memory (but calculating says something different: 10 radio streams: ~2.5GB ; 75 Tonies = ~50MBx75= ~3,75GB ; ~total: 6,25GB)

The logs tells:

removing a tonie with radio stream:

INFO |toniefile.c:0783:ffmpeg_stream| Encoding aborted, active flag set to false
Error writing trailer of pipe:: Broken pipe
av_interleaved_write_frame(): Broken pipe
size=   32694kB time=00:02:54.39 bitrate=1535.8kbits/s speed=1.12x    
Error closing file pipe:: Broken pipe
video:0kB audio:32699kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!
INFO |toniefile.c:0627:ffmpeg_decode_audio_end| Stopped ffmpeg with code=256...
ERROR|toniefile.c:0800:ffmpeg_stream| Could not close FFmpeg pipe error=Operation aborted [15]
INFO |toniefile.c:0815:ffmpeg_stream| TAF encoding stopped (streaming)

Placing a Tonie with radio stream assigned without doing a freshness check:

INFO |toniefile.c:0783:ffmpeg_stream| Encoding aborted, active flag set to false
av_interleaved_write_frame(): Broken pipe
Error writing trailer of pipe:: Broken pipe
size=    9788kB time=00:00:52.22 bitrate=1535.4kbits/s speed=1.22x    
video:0kB audio:9792kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Error closing file pipe:: Broken pipe
Conversion failed!
INFO |toniefile.c:0627:ffmpeg_decode_audio_end| Stopped ffmpeg with code=256...
ERROR|toniefile.c:0800:ffmpeg_stream| Could not close FFmpeg pipe error=Operation aborted [15]
=================================================================
==8==ERROR: AddressSanitizer: heap-use-after-free on address 0x007f526e6480 at pc 0x00556c188a9c bp 0x007f505948f0 sp 0x007f50594938
READ of size 3 at 0x007f526e6480 thread T99
    #0 0x556c188a98 in __interceptor_fopen.part.0 (/usr/local/bin/teddycloud+0x648a98)
    #1 0x556c62fbfc in fsOpenFile cyclone/common/fs_port_posix.c:277
    #2 0x556c25fdc8 in isValidTaf src/handler.c:477
    #3 0x556c311a20 in toniefile_close src/toniefile.c:365
    #4 0x556c319da0 in ffmpeg_stream src/toniefile.c:807
    #5 0x556c31d4b4 in ffmpeg_stream_task src/toniefile.c:833
    #6 0x7f9ab7ee8c  (/lib/aarch64-linux-gnu/libc.so.6+0x7ee8c)
    #7 0x7f9abe7b18  (/lib/aarch64-linux-gnu/libc.so.6+0xe7b18)
0x007f526e6480 is located 0 bytes inside of 58-byte region [0x007f526e6480,0x007f526e64ba)
freed by thread T66 here:
    #0 0x556c1d4020 in __interceptor_free.part.0 (/usr/local/bin/teddycloud+0x694020)
    #1 0x556c23383c in free_content_json src/contentJson.c:289
    #2 0x556c2651dc in freeTonieInfo src/handler.c:784
    #3 0x556c29df98 in handleCloudContent src/handler_cloud.c:742
    #4 0x556c2d8680 in httpServerRequestCallback src/server.c:483
    #5 0x556c42dfe0 in httpConnectionTask src/cyclone/cyclone_tcp/http/http_server.c:541
    #6 0x7f9ab7ee8c  (/lib/aarch64-linux-gnu/libc.so.6+0x7ee8c)
    #7 0x7f9abe7b18  (/lib/aarch64-linux-gnu/libc.so.6+0xe7b18)
previously allocated by thread T66 here:
    #0 0x556c1d51b4 in __interceptor_malloc (/usr/local/bin/teddycloud+0x6951b4)
    #1 0x556c2e2754 in custom_asprintf src/server_helpers.c:40
    #2 0x556c231678 in load_content_json src/contentJson.c:24
    #3 0x556c262db8 in getTonieInfo src/handler.c:666
    #4 0x556c264c10 in getTonieInfoFromRuid src/handler.c:635
#5 0x556c29d720 in handleCloudContent src/handler_cloud.c:474
    #6 0x556c2d8680 in httpServerRequestCallback src/server.c:483
    #7 0x556c42dfe0 in httpConnectionTask 
rc/cyclone/cyclone_tcp/http/http_server.c:541
    #8 0x7f9ab7ee8c  (/lib/aarch64-linux-gnu/libc.so.6+0x7ee8c)
    #9 0x7f9abe7b18  (/lib/aarch64-linux-gnu/libc.so.6+0xe7b18)
Thread T99 created by T66 here:
    #0 0x556c173cd4 in __interceptor_pthread_create (/usr/local/bin/teddycloud+0x633cd4)
    #1 0x556c62ea48 in osCreateTask src/cyclone/common/os_port_posix.c:87
    #2 0x556c29e780 in handleCloudContent src/handler_cloud.c:606
    #3 0x556c2d8680 in httpServerRequestCallback src/server.c:483
    #4 0x556c42dfe0 in httpConnectionTask src/cyclone/cyclone_tcp/http/http_server.c:541
    #5 0x7f9ab7ee8c  (/lib/aarch64-linux-gnu/libc.so.6+0x7ee8c)
    #6 0x7f9abe7b18  (/lib/aarch64-linux-gnu/libc.so.6+0xe7b18)
Thread T66 created by T0 here:
    #0 0x556c173cd4 in __interceptor_pthread_create (/usr/local/bin/teddycloud+0x633cd4)
    #1 0x556c62ea48 in osCreateTask src/cyclone/common/os_port_posix.c:87
    #2 0x556c42ab4c in httpServerStart src/cyclone/cyclone_tcp/http/http_server.c:233
    #3 0x556c2dfb7c in server_init src/server.c:874
    #4 0x556c14c248 in main src/main.c:610
    #5 0x7f9ab2773c  (/lib/aarch64-linux-gnu/libc.so.6+0x2773c)
    #6 0x7f9ab27814 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x27814)
    #7 0x556c1519ac in _start (/usr/local/bin/teddycloud+0x6119ac)
SUMMARY: AddressSanitizer: heap-use-after-free (/usr/local/bin/teddycloud+0x648a98) in _interceptor_fopen.part.0
Shadow bytes around the buggy address:
  0x001fea4dcc40: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd
  0x001fea4dcc50: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
  0x001fea4dcc60: fd fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
  0x001fea4dcc70: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd
  0x001fea4dcc80: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
=>0x001fea4dcc90:[fd]fd fd fd fd fd fd fd fa fa fa fa fd fd fd fd
  0x001fea4dcca0: fd fd fd fa fa fa fa fa fd fd fd fd fd fd fd fa
  0x001fea4dccb0: fa fa fa fa fd fd fd fd fd fd fd fa fa fa fa fa
  0x001fea4dccc0: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd
  0x001fea4dccd0: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fd
  0x001fea4dcce0: fa fa fa fa fd fd fd fd fd fd fd fd fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==8==ABORTING
Running teddycloud...
TeddyCloud v0.6.4 (f13703a) - 2025-03-05 08:01:37 +0000 debian linux-aarch64(64)
INFO |settings.c:0860:settings_load_ovl| Load settings from teddycloud/config/config.overlay.ini
INFO |tls_adapter.c:0204:read_certificate| File '/teddycloud/certs/server/ca-root.pem' ssumed PEM style
INFO |tls_adapter.c:0201:read_certificate| File '/teddycloud/certs/server/ca-key.pem' detected as DER style RSA PRIVATE KEY
INFO |tls_adapter.c:0204:read_certificate| File '/teddycloud/certs/server/teddy-cert.pem' assumed PEM style
INFO |tls_adapter.c:0201:read_certificate| File '/teddycloud/certs/server/teddy-key.pem' detected as DER style RSA PRIVATE KEY
ERROR|tls_adapter.c:0189:read_certificate| Failed to open '/teddycloud/certs/client/f4f5bd64b8e4/ca.der' for cert type detection
ERROR|tls_adapter.c:0376:load_cert| Loading cert '/teddycloud/certs/client/f4f5bd64b8e4/ca.der' failed
ERROR|tls_adapter.c:0189:read_certificate| Failed to open '/teddycloud/certs/client/f4f5bd64b8e4/client.der' for cert type detection
ERROR|tls_adapter.c:0376:load_cert| Loading cert '/teddycloud/certs/client/f4f5bd64b8e4/client.der' failed
ERROR|tls_adapter.c:0189:read_certificate| Failed to open '/teddycloud/certs/client/f4f5bd64b8e4/private.der' for cert type detection
ERROR|tls_adapter.c:0376:load_cert| Loading cert '/teddycloud/certs/client/f4f5bd64b8e4/private.der' failed
INFO |settings.c:0860:settings_load_ovl| Load settings from /teddycloud/config/config.ini
INFO |tls_adapter.c:0204:read_certificate| File '/teddycloud/certs/server/ca-root.pem' assumed PEM style
INFO |tls_adapter.c:0201:read_certificate| File '/teddycloud/certs/server/ca-key.pem' detected as DER style RSA PRIVATE KEY
INFO |tls_adapter.c:0204:read_certificate| File '/teddycloud/certs/server/teddy-cert.pem' ssumed PEM style
INFO |tls_adapter.c:0201:read_certificate| File '/teddycloud/certs/server/teddy-key.pem' detected as DER style RSA PRIVATE KEY
INFO |tls_adapter.c:0197:read_certificate| File '/teddycloud/certs/client/ca.der' detected as DER style CERTIFICATE
INFO |tls_adapter.c:0197:read_certificate| File '/teddycloud/certs/client/client.der' detected as DER style CERTIFICATE
INFO |tls_adapter.c:0201:read_certificate| File '/teddycloud/certs/client/private.der' detected as DER style RSA PRIVATE KEY
INFO |settings.c:0860:settings_load_ovl| Load settings from /teddycloud/config/config.overlay.ini
INFO |tls_adapter.c:0204:read_certificate| File '/teddycloud/certs/server/ca-root.pem' assumed PEM style
INFO |tls_adapter.c:0201:read_certificate| File '/teddycloud/certs/server/ca-key.pem' detected as DER style RSA PRIVATE KEY
INFO |tls_adapter.c:0204:read_certificate| File '/teddycloud/certs/server/teddy-cert.pem' assumed PEM style
INFO |tls_adapter.c:0201:read_certificate| File '/teddycloud/certs/server/teddy-key.pem' detected as DER style RSA PRIVATE KEY
ERROR|tls_adapter.c:0189:read_certificate| Failed to open '/teddycloud/certs/client/f4f5bd64b8e4/ca.der' for cert type detection
ERROR|tls_adapter.c:0376:load_cert| Loading cert /teddycloud/certs/client/f4f5bd64b8e4/ca.der' failed
ERROR|tls_adapter.c:0189:read_certificate| Failed to open '/teddycloud/certs/client/f4f5bd64b8e4/client.der' for cert type detection
ERROR|tls_adapter.c:0376:load_cert| Loading cert '/teddycloud/certs/client/f4f5bd64b8e4/client.der' failed
ERROR|tls_adapter.c:0189:read_certificate| Failed to open '/teddycloud/certs/client/f4f5bd64b8e4/private.der' for cert type detection
ERROR|tls_adapter.c:0376:load_cert| Loading cert '/teddycloud/certs/client/f0f5bd64b8e4/private.der' failed
INFO |tls_adapter.c:0390:tls_adapter_init| Loading certificates...
INFO |tls_adapter.c:0204:read_certificate| File '/teddycloud/certs/server/ca-root.pem' assumed PEM style
INFO |tls_adapter.c:0201:read_certificate| File '/teddycloud/certs/server/ca-key.pem' detected as DER style RSA PRIVATE KEY
INFO |tls_adapter.c:0204:read_certificate| File '/teddycloud/certs/server/teddy-cert.pem' assumed PEM style
INFO |tls_adapter.c:0201:read_certificate| File '/teddycloud/certs/server/teddy-key.pem' detected as DER style RSA PRIVATE KEY
INFO |tls_adapter.c:0197:read_certificate| File '/teddycloud/certs/client/ca.der' detected as DER style CERTIFICATE
INFO |tls_adapter.c:0197:read_certificate| File '/teddycloud/certs/client/client.der' detected as DER style CERTIFICATE
INFO |tls_adapter.c:0201:read_certificate| File '/teddycloud/certs/client/private.der' detected as DER style RSA PRIVATE KEY
INFO |toniesJson.c:0280:tonies_readJson| Trying to read /teddycloud/config/tonies.custom.json with size 3459
INFO |toniesJson.c:0280:tonies_readJson| Trying to read /teddycloud/config/tonies.json with size 5638426
INFO |toniesJson.c:0100:tonies_update| Updating tonies.json from api.revvox.de...
INFO |cloud_request.c:0200:web_request| Connecting to HTTP server api.revvox.de:443...
INFO |cloud_request.c:0252:web_request|   trying IP: xxx.xx.xxx.xxx
INFO |cloud_request.c:0389:web_request| Redirecting to: https://raw.githubusercontent.com/toniebox-reverse-engineering/tonies-json/release/tonies.json
INFO |cloud_request.c:0200:web_request| Connecting to HTTP server raw.githubusercontent.com:443...
INFO |cloud_request.c:0252:web

Any idea?
Wondering also what happened with the certificate?

HardwareJunkie · September 18, 2025, 10:51pm

Error Description

The log excerpt shows a series of errors related to the teddycloud software. The main error causing the crash is a “heap-use-after-free” problem. This is a critical programming bug where the software tries to access (read from) a memory area that has already been deallocated.

Detailed Log Analysis

FFmpeg Errors: The first few lines indicate problems with audio/video processing using the FFmpeg library.
- Encoding aborted, active flag set to false
- av_interleaved_write_frame(): Broken pipe
- Error writing trailer of pipe:: Broken pipe
- Conversion failed! These messages suggest that a conversion or streaming process running in a separate process (pipe) was terminated unexpectedly. This could be a side effect of the subsequent memory error.
AddressSanitizer Error: This is the most crucial part of the message that explains the crash.
- ==8==ERROR: AddressSanitizer: heap-use-after-free on address 0x007f526e6480
- READ of size 3 at 0x007f526e6480 thread T99
- freed by thread T66 here:
- previously allocated by thread T66 here: This is a detailed analysis provided by a debugging tool called AddressSanitizer (ASan). It tells us the following:
- A memory corruption error (heap-use-after-free) occurred.
- It happened at memory address 0x007f526e6480.
- A different thread (T66) had previously allocated this memory and then freed it.
- Subsequently, a third thread (T99) tried to access the same now-invalid memory area (READ of size 3), which led to the program’s crash.
TLS/Certificate Errors:
- ERROR|tls_adapter.c...Failed to open...
- Loading cert...failed These errors show that the software is trying to load specific certificate files (ca.der, client.der, private.der), but it’s failing because the files are not found in the expected locations. These are configuration issues that might prevent the software from connecting to certain online services.
Update Error:
- INFO |toniesJson.c:0100:tonies_update| Updating tonies.json from api.revvox.de...
- INFO |cloud_request.c...Redirecting to: https://raw.githubusercontent.com...
- INFO |cloud_request.c...Connecting to HTTP server raw.githubusercontent.com:443... The software is attempting to update a file (tonies.json) from a server but is being redirected. This is not an error in itself, but the process appears to be incomplete as the log cuts off shortly after.

Summary

The main cause of the crash is a severe software bug (heap-use-after-free), likely caused by a race condition between different threads (T66 and T99). While one thread frees the memory, another tries to use it, which results in the program terminating.

The other messages about “Broken pipe” and missing certificates are likely either follow-on errors or separate configuration issues that don’t directly cause the crash but point to a potentially unstable environment or faulty setup. You should report this log output to the software developer.