Author Topic: \0 terminated SOCK_RECV command  (Read 5906 times)

Offline FutureCow

  • HelpEditor
  • Prof. Inline
  • ******
  • Posts: 680
    • View Profile
\0 terminated SOCK_RECV command
« on: 2010-Jan-21 »
The original SOCK_RECV() would chop your messages up nicely by stopping reading when a '\0' was read.
This was changed after I found I couldn't transmit binary files due to \0 characters through them.

Now I've found I could really do with two versions of the SOCK_RECV command.
1) The current one that will read \0 as part of the stream (for binary transfers)
2) One that terminates at \0 for non-binary transfers (i.e. the way SOCK_RECV used to work)

Currently if you send something like
Code: GLBasic [Select]
SOCK_TCPSEND([socket],"SetLives=1")
SOCK_TCPSEND([socket],"SetLevel=14")
SOCK_TCPSEND([socket],"ResetHighScoreTable")

then do a
Code: GLBasic [Select]
SOCK_RECV([socket],LivesString$,1023)
SOCK_RECV([socket],LevelString$,1023)
SOCK_RECV([socket],ResetString$,1023)
 
InputString will contain "SetLives=1SetLevel=14ResetHighScoreTable" and LevelString$/ResetString$ will probably be empty. And it's not always consistent that all the output will be in the first SOCK_RECV() output as it depends on the timing when you run the SOCK_RECV command.

Having a new command that terminates on the '\0' means I could do 3 separate SOCK_RECV() calls and get each parameter and its setting separately for processing. Without it, I'm currently having to do something similar to the following (this is just an example, mines slightly more efficient)
Code: GLBasic [Select]
Server
=======
SOCK_TCPSEND([socket],"SetLives=1")

Client
======
while Message$=""
    SOCK_RECV([sock],Message$,1023)
wend
if Message$="SetLives"
    SOCK_TCPSEND([socket],"Acknowledged")

Server
=======
while Message$=""
    SOCK_RECV([sock],Message$,1023)
wend
if Message$="Acknowledged"
    SOCK_TCPSEND([socket],"SetLevel=14")

[...]
 
and so on which wastes time waiting for acknowledgements and causes excess network chatter, or some really annoying string manipulations to strip each different length string off the network stream.

Offline Kitty Hello

  • code monkey
  • Administrator
  • Prof. Inline
  • *******
  • Posts: 10653
  • here on my island the sea says 'hello'
    • View Profile
    • http://www.glbasic.com
Re: \0 terminated SOCK_RECV command
« Reply #1 on: 2010-Jan-21 »
You get an ascii stream that has \0 characters? What is that?
If you know that, you could try SPLITSTR(str$, chr$(0));

Offline FutureCow

  • HelpEditor
  • Prof. Inline
  • ******
  • Posts: 680
    • View Profile
Re: \0 terminated SOCK_RECV command
« Reply #2 on: 2010-Jan-21 »
I thought from testing the old version that the tcp_send automatically inserted a "\0" at the end of each transmission string.   :noggin:  I assume from your reply that I'm probably incorrect though.

SPLITSTR isn't quite as neat, but it'll be easier than my current solution - thanks Gernot!

Offline Kitty Hello

  • code monkey
  • Administrator
  • Prof. Inline
  • *******
  • Posts: 10653
  • here on my island the sea says 'hello'
    • View Profile
    • http://www.glbasic.com
Re: \0 terminated SOCK_RECV command
« Reply #3 on: 2010-Jan-21 »
now wait - the TCP_SEND will _not_ append a '\0' character, because web browsers don't send that, either.
If you receive, I will always force a '\0' at the end of what I read, so you can have a const char* to that string. With GLBasic only, you should not worry, because I have a length variable for each sting.
If you receive a string that contains a '\0' character, however, I'd be glad to hear where that comes from. Don't get me wrong, I'm not saying you do something wrong. It might easily be me missunderstanding of the specs.

Offline FutureCow

  • HelpEditor
  • Prof. Inline
  • ******
  • Posts: 680
    • View Profile
Re: \0 terminated SOCK_RECV command
« Reply #4 on: 2010-Jan-21 »
Okay now I'm getting confused!  :D
My problem was that I didn't know in advance (before changing my code anyway) how long a string I was going to be sending at various stages. I assumed I could send any string and a '\0' would automatically append. Therefore I thought I could send say 5 strings of various lengths (say <100 characters), and (in theory) do 5 sock_recv's of 1023 characters to read them and it would all work nicely.

I just made the assumption from what I saw that the strings would automatically have a '\0' on them from TCP_SEND.
I'll have to have a look tomorrow and see how people normally do efficient stream communication for separate messages they want to send. I'm currently sending a 8 char "header" and my code knows for each different that header I send how many chars follow. I then use MID$ to delete the total number of processed chars from the string, and reprocess the string until it's empty. It's not neat but the best solution to fit my needs at short notice.

Offline Kitty Hello

  • code monkey
  • Administrator
  • Prof. Inline
  • *******
  • Posts: 10653
  • here on my island the sea says 'hello'
    • View Profile
    • http://www.glbasic.com
Re: \0 terminated SOCK_RECV command
« Reply #5 on: 2010-Jan-21 »
OK. Here's what I think about that. I have no idea if I'm right:
- You send data to another computer.
  - I send() as much of that as I can send.
  - if there's more data, I send again and again

- You read from a socket
  -I read until the other end either disconnects, or there is nothing more to read (EWOULDBLOCK)

Can anyone shed light on this? Ocean, maybe?



Offline Moru

  • Administrator
  • Prof. Inline
  • *******
  • Posts: 1750
    • View Profile
    • Homepage
Re: \0 terminated SOCK_RECV command
« Reply #6 on: 2010-Jan-21 »
One thing I have always wondered about is if my app sends data very fast in small packets, will I still get the packets one at a time when I read from the socket or will they be concatenated on the way and sent as one packet? I was using some library for Visual Basic a few years ago and in the documentation it said not to expect the packets to stay separated, they could get concatenated in transfer. You had to put your own start packet/end packet codes into your packets.

About the original question, a packet is binary data, I don't think it should stop reading on 0, you should read until EOF or similar.
« Last Edit: 2010-Jan-21 by Moru »

Offline Moru

  • Administrator
  • Prof. Inline
  • *******
  • Posts: 1750
    • View Profile
    • Homepage
Re: \0 terminated SOCK_RECV command
« Reply #7 on: 2010-Jan-21 »
I have tested this long time ago (before the new net commands) and there was no need for zero-termination. I used \0 as split between command and parameters instead since glbasic don't terminate a string on ascii 0. Only thing that is important is the length of the string, just as it's supposed to be in basic. Zero-termination is just there for compatibility with inline C I guess?

Offline Kitty Hello

  • code monkey
  • Administrator
  • Prof. Inline
  • *******
  • Posts: 10653
  • here on my island the sea says 'hello'
    • View Profile
    • http://www.glbasic.com
Re: \0 terminated SOCK_RECV command
« Reply #8 on: 2010-Jan-21 »
OK, I got enlightened now.

When you send strings (long ones) over the internet, usually you provide some sort of "header" for you data. That means, you read 4 bytes that describe the length of the coming package, and then read this length from the socket, checking for errors (connection broken e.g.)

If you read from a website or so, the thing acts differently. They have a sort of EOF by disconnecting after the sending. The server will send you the full page and then close its socket. That way you know that your data was received completely.

For fast interchange, I'd suggest going route #1. That's BTW the thing I do with the NETSENDMSG commands.

Did I miss a thing?

Offline FutureCow

  • HelpEditor
  • Prof. Inline
  • ******
  • Posts: 680
    • View Profile
Re: \0 terminated SOCK_RECV command
« Reply #9 on: 2010-Jan-22 »
Here's a test program with some interesting results

NOTE: Compile as a console program then run 2 copies of it.
Code: GLBasic [Select]
GLOBAL Net_ServerIP$
GLOBAL EstablishedConnection
GLOBAL Net_SocketNumber
GLOBAL Done
Net_SocketNumber=1234
Net_ServerIP$="127.0.0.1"

SOCK_INIT()

// MAIN
//-------------------------------------------------------------------------
Done=0

STDOUT "Press 1 to be server\n"
STDOUT "Press 2 to be client\n"

LOCAL Result$=INKEY$()
IF Result$ = "1"                // pressed 1
        STDOUT "Server\n"
        DEBUG "Server-debug\n"
        GOSUB server
ELSEIF Result$= "2"     // pressed 2
        STDOUT "Client\n"
        DEBUG "Client-debug\n"
        GOSUB Client
ENDIF


//-------------------------------------------------------------------------
SUB server:
        LOCAL Done=0
        LOCAL ClientIPAddress%
        LOCAL Sock%
        STDOUT "Waiting for connection...\n"
       
        Sock=SOCK_TCPLISTEN(Net_SocketNumber)                                                                                           // Listen for connections on our socket
       
        WHILE Done=0
                EstablishedConnection=SOCK_TCPACCEPT(Sock,ClientIPAddress)                                      // Check for established client connection
                IF EstablishedConnection > 0 THEN Done=1                                                                        // If a connection was successful
        WEND
       
        // We now have a connection
       
        SOCK_TCPSEND(EstablishedConnection,"1")
        SOCK_TCPSEND(EstablishedConnection,"2\0")
        SOCK_TCPSEND(EstablishedConnection,"3"+CHR$(0))
        SOCK_TCPSEND(EstablishedConnection,"4")
        SOCK_TCPSEND(EstablishedConnection,"5\0")
        SOCK_TCPSEND(EstablishedConnection,"6"+CHR$(0))
       
        DEBUG ("Server Done\n")
        SLEEP 10000
ENDSUB

//-------------------------------------------------------------------------
SUB Client:
        LOCAL Done=0
        LOCAL ReturnCode
        LOCAL ReceivedMessage$
        LOCAL YLoc=50
       
        STDOUT "Establishing connection with "+Net_ServerIP$+" on port "+Net_SocketNumber+"...\n"
        EstablishedConnection=SOCK_TCPCONNECT(Net_ServerIP$,Net_SocketNumber)           // Try to connect to server
        IF EstablishedConnection = -1                                                                           // Failure
                STDERR "Network error : client failed to create connection"
                END
        ENDIF
       
        SLEEP 500               // allow messages to cache
        LOCAL Done=0
        WHILE Done=0
                ReturnCode=SOCK_RECV(EstablishedConnection, ReceivedMessage$, 1023)
                IF ReturnCode=0
                        STDERR "Remote end has disconnected\n"
                        END
                ELSEIF ReturnCode=-1
                        STDERR "Network error has occurred"
                        STDERR "Error received was : "+NETGETLASTERROR$()+"\n"
                        END
                ELSEIF ReturnCode=-2                    // Nothing waiting in the queue
                        Done=1
                ELSE
                        DEBUG "Message length received : "+ReturnCode+"   Message recieved = " + ReceivedMessage$ + "\n"
                        STDOUT "Message length received : "+ReturnCode+"   Message recieved = " + ReceivedMessage$ + "\n"
                ENDIF
        WEND
        SLEEP 10000
ENDSUB
       
       

What you will see is that the client receives only one message - the output from the client is
Code: GLBasic [Select]
Message length received : 8   Message recieved = 123

4,5 and 6 don't appear, as the CHR$(0) terminates the printing of the string, but you can tell by the message length (8 characters) that all the data was sent, then returned by one SOCK_RECV() command.

From everything I've found doing an internet search, it appears that all characters (including \0) should be sent with send() or recv() - the use of the NUL character as a message delimeter would be entirely a user implementation issue.

From what you wrote Gernot I assume that NETGETMSG and NETSENDMSG are appending a \0 and doing a SPLITSTR to separate the messages?

Offline Kitty Hello

  • code monkey
  • Administrator
  • Prof. Inline
  • *******
  • Posts: 10653
  • here on my island the sea says 'hello'
    • View Profile
    • http://www.glbasic.com
Re: \0 terminated SOCK_RECV command
« Reply #10 on: 2010-Jan-22 »
It acts as expected. You send data and the other side can read data.
So, if you want to send a string and do _not_ close the connection but want to write another string, you must append a string length header. That's what I do with the NET commands.

Here's what you want:
Code: GLBasic [Select]
// --------------------------------- //
// Project: _moo
// Start: Friday, January 22, 2010
// IDE Version: 7.242


        // SETCURRENTDIR("Media") // seperate media and binaries?
GLOBAL Net_ServerIP$
GLOBAL EstablishedConnection%
GLOBAL Net_SocketNumber%
Net_SocketNumber=1234
Net_ServerIP$="127.0.0.1"

SOCK_INIT()

// MAIN
//-------------------------------------------------------------------------
Done=0

STDOUT "Press 1 to be server\n"
STDOUT "Press 2 to be client\n"

LOCAL Result$=INKEY$()
IF Result$ = "1"      // pressed 1
   STDOUT "Server\n"
   GOSUB server
ELSEIF Result$= "2"   // pressed 2
   STDOUT "Client\n"
   GOSUB Client
ENDIF


FUNCTION SendString%: sock%, str$
        // pack the length as a 10 byte string
        SOCK_TCPSEND(sock%, FORMAT$(10,0,LEN(str$)) + str$)
ENDFUNCTION


FUNCTION ReceiveString$: sock%
LOCAL size%, str$, ok%
        ok% = SOCK_RECV(sock%, str$, 10) // length package
        IF ok% <= 0 THEN RETURN "" // nothing there
       
        // now we must read the string - dead or alive, you&#39;re sending to me
        size% = str$
        // no need to loop - GLBasic does that for you
        WHILE TRUE
                ok% = SOCK_RECV(sock%, str$, size%)
                IF ok% = -2 THEN CONTINUE // no data sent, yet. Retry
                BREAK
        WEND
        RETURN str$
ENDFUNCTION



//-------------------------------------------------------------------------
SUB server:
   LOCAL Done=0
   LOCAL ClientIPAddress%
   LOCAL Sock%
   STDOUT "Waiting for connection...\n"
   
   Sock=SOCK_TCPLISTEN(Net_SocketNumber)                                    // Listen for connections on our socket
   
   WHILE Done=0
      EstablishedConnection=SOCK_TCPACCEPT(Sock,ClientIPAddress)               // Check for established client connection
      IF EstablishedConnection > 0 THEN Done=1                           // If a connection was successful
   WEND
   
   // We now have a connection
   STDOUT "Sending...\n"
   
   LOCAL names$[]
   DIMDATA names$[], "John Rambo", "Jim Knopf", "Kermit the Frog", "Dr. Shiver - buy now"
   
   WHILE TRUE
      SendString(EstablishedConnection, names$[RND(LEN(names$[])-1)] )
      SLEEP 3000
   WEND
ENDSUB

//-------------------------------------------------------------------------
SUB Client:
   LOCAL Done=0
   LOCAL ReturnCode
   LOCAL ReceivedMessage$
   LOCAL YLoc=50
   
   STDOUT "Establishing connection with "+Net_ServerIP$+" on port "+Net_SocketNumber+"...\n"
   EstablishedConnection=SOCK_TCPCONNECT(Net_ServerIP$,Net_SocketNumber)      // Try to connect to server
   IF EstablishedConnection = -1                              // Failure
      STDERR "Network error : client failed to create connection"
      END
   ENDIF

   WHILE TRUE
       ReceivedMessage$ = ReceiveString$(EstablishedConnection)
       IF LEN(ReceivedMessage$)
               STDOUT "Got: \"" + ReceivedMessage$ + "\" len="+LEN(ReceivedMessage$)+"\n"
           ENDIF
       SLEEP 500
   WEND

ENDSUB
 

Offline Kitty Hello

  • code monkey
  • Administrator
  • Prof. Inline
  • *******
  • Posts: 10653
  • here on my island the sea says 'hello'
    • View Profile
    • http://www.glbasic.com
Re: \0 terminated SOCK_RECV command
« Reply #11 on: 2010-Jan-22 »
Read it into a string then send. The LEN and SEND functions do not care about '\0' characters.

Offline Kitty Hello

  • code monkey
  • Administrator
  • Prof. Inline
  • *******
  • Posts: 10653
  • here on my island the sea says 'hello'
    • View Profile
    • http://www.glbasic.com
Re: \0 terminated SOCK_RECV command
« Reply #12 on: 2010-Jan-22 »
the UUENCODE is something you have to do yourself. The socket commands don't do that.
I have posted an article how to upload a png file to a webserver (HTTP), see that.

I think all problems are solved now, no?

Offline Moru

  • Administrator
  • Prof. Inline
  • *******
  • Posts: 1750
    • View Profile
    • Homepage
Re: \0 terminated SOCK_RECV command
« Reply #13 on: 2010-Jan-22 »
I don't realy see the problem, GLB treats strings as binary data right up to the instant you try to print it to the screen, right? So just use it as binary data, don't print it to the screen.