I had a similar problem. Sometimes I could establish a connection, but most of the time it timed out. My solution: I used resistor wires to connect to the UART. These were shortened to a maximum of 3cm without any additional cables. The UART is then practically soldered to the board, and you have to be careful not to create a short circuit (when crossing the pins). I hope it works for you.
Ok, after wasting so many days without any progress, I give up now.
I bought an ESP32 version, which worked with the same setup directly on the first try.