As you look at the schematic, note that there are two 5 volt rails and two grounds. Nothing is allowed to cross over the isolation barrier at all (except light inside the isolators). We need to move 3 signals in each direction. The SI8035 is a logic level isolator triplet, so we need 2 of them for 6 signals (3 in each direction). Take careful note that one of the isolator chips is upside down relative to the other one. That's because each host needs one primary and one secondary side.
The inputs and outputs of the isolators are just simple logic level I/O that's compatible with the I/O lines on the UART chip. The tricky part is in understanding that the secondary side shares the 5 volt and ground of the host it talks to, but the matching primary side shares its 5 volt and ground with the other host.
The wiring of the null modem is TXD->RXD, CTS->RTS and DTR->DSR+DCD, which is a fairly standard configuration.
The USB UART chips are the Cypress CY7C65213. They present a standard CDC interface over USB. The USB connectors are USB C 2.0 types. They can connect to USB C hosts or to legacy hosts via an A->C cable.
The last note is that you should take careful note of the fact that the groundplane for each host stops just beyond the line of isolator pins. Under the isolators there is no copper at all. The SI8035 has an isolation voltage spec of 1 kV and the two ground planes are 6.3 mm or so apart, which gives you up to 2 kV, depending on what pollution degree you choose. so the lower of the two - 1 kV is the isolation spec for the project as a whole. Still, for the ordinary use case, this should be plenty. Two hosts close enough together to reach via USB that have a ground potential difference more than a handful of volts would already be a fairly dangerous situation.
The USB bridge has a max data rate of 3Mbps and the opto isolator is max 20Mbps. What was the max data rate you could achieve during testing?