Has anyone got a better solution for transmitting files than
SENDING END
* Do 1023 READUBYTE commands, and append the result each time to a transmission string
* send the string via NETSENDMSG.
* Write some sort of EOF string to the buffer
RECEIVING END
* receive the string with NETGETMSG
* Append the string to the destination file with WRITEUBYTE
* stop reading when EOF string read
I haven't tried it yet, but I'm not a fan of trying to encode a binary file as a string - particularly as I think I'm likely to run into typecasting errors.
Any thoughts anyone?
I'm not sure but I think I tried sending data as binary over the network... I'll try to dig out the code again and test it with the new netcode. Only trouble I can imagine is the byte 0, the rest is just a character in the chartable so should work.
You can put chr$(0) in a string in GLBasic!! You can have binary data in a string.
If you transfer it over the network (use SOCK_ here) you'd better write a tiny header (filetype and length) and then the data.
I hadn't looked at SOCK_ yet, I'd been looking at NET_
What's the maximum length (would be handy to have in the docs :whip: ;) ) that you can send with SOCK_TCPSEND?
sock_tcpsend has no limit. The NET commands are limited to ... uhm... 1024 bytes.. maybe?
So is there any reason I wouldn't want to send say a 1MB file as 1 SOCK_TCPSEND command? What would you consider "best practice?"
Though I suppose doing this there would be no way to show how much has been transferred (ie. 1% done, 2% done ... etc)
There is a limit on how big packets some routers can transfer, keep the size below 1540 at least, 1024 is probably safer :)
Take a look at my Network for cowards tutorials, where I upload a file to a webserver running php. That's exaclty what you want.
Not all routers can chop up packets reliably, that's why I say to keep the size down if you are not programming only for yourself.
Are you planning on transfering between two GLBasic applications on the same network or do you want to transfer over the internet to a webserver for access later or what are you trying to do?
Just for info:
You are talking about MTU ( Maximum transmission unit).
And as was said, the MTU for Ethernet v2 is 1500 bytes.
Just for compatibility issues and safety usually is configured to 1492 bytes that is the MTU for Ethernet (802.3).
Yes but most equipment can handle bigger by splitting packets and assembling these on the other hand. This should all be taken care of outside of our control.
Yes.
Thi bigger packets, out of rfc, as far as I can remember, are called jumbo frames.
But as you said, only should be used on controlled environments.
I'm actually a Unix admin by trade, so I'm well aware of MTUs, packet reassembly etc. What I'm not aware of is the black magic happening in the background (ie. what GLBasic actually does when you use that command) - hence the question. There would be no point chopping your data into chunks if the command was doing that itself in the background.
I'll stick to 1024 bytes I think, that's a convenient number, and besides which it makes it easy to do a "number of KB uploaded" counter.
Kitty Hello/Moru : I had a look at your tutorials (they're what I've been working off) but they deal with transmitting files to a webserver. I'm doing GLBasic application to another copy of the same program elsewhere on the network/internet (ie. sharing game data between two people playing the same game). There's no reason I can see that players should have to run a webserver.
I have a webserver (mini mini) in GLBasic, somewhere, too... Wait..
// Simple "web" server program to interact with a browser
GLOBAL WEB_port% = 11111
SOCK_INIT()
LOCAL sock_listen% = SOCK_TCPLISTEN(WEB_port%)
WHILE TRUE
LOCAL cl_ip%
LOCAL client% = SOCK_TCPACCEPT(sock_listen%, cl_ip%)
IF client% <> -1
STDOUT "Connection\n"
LOCAL rv%, msg$
LOCAL full$
WHILE TRUE
rv% = SOCK_RECV(client%, msg$, 1024)
IF rv = 0 THEN GOTO client_quit
INC full$, msg$
IF rv%<1024 THEN BREAK
WEND
STDOUT "Full Message:\n" + full$+"\n\n"
SendMessage(client%, "<html><body><h1>HELLO WORLD</H1></body></html>")
SOCK_CLOSE(client) // no keep-alive connection, K
ENDIF
client_quit:
WEND
FUNCTION Http_TextArea: varname$
// TODO
ENDFUNCTION
FUNCTION SendMessage: sock%, postdata$
LOCAL send$
send$ = "HTTP/1.0 200 OK\r\n"
INC send$, "Date: "+PLATFORMINFO$("DATE")+" "+PLATFORMINFO$("TIME")+"\r\n"
INC send$, "Content-Type: text/html\r\n"
INC send$, "Content-Length: "+LEN(postdata$) + "\r\n"
INC send$, "\r\n"
INC send$, postdata$
SOCK_TCPSEND(sock%, send$)
ENDFUNCTION
No magic.
NO magic? :nw: EVERYTHING you do is magic Gernot! =D
Anyone have any ideas? I've tried this both reading strings and reading UBytes, and the client seems to stop reading data when it hits a \0 value. When I get this sorted I'll post it as a code example on the forum.
The source code wis set up to run on one pc (ie. IP address 127.0.0.1) - you'll need to run 2 copies, one for the client, one for the server.
Change the SourceFile to reflect a valid file on your Server system, and DestFile to where you'd like the copy put on the Client.
GLOBAL SourceFile$
GLOBAL DestFile$
SourceFile$="c:/glbasic/outfile.txt"
DestFile$="c:/glbasic/infile.txt"
// ------------------------------------------------------------- //
// --- MAIN ---
// ------------------------------------------------------------- //
SOCK_INIT()
LOCAL quit=0
WHILE quit=0
STDOUT "(C)lient / (S)erver / (Q)uit\n"
Result$ = INKEY$()
IF Result$="C" OR Result$="c"
GOSUB ClientSub
ENDIF
IF Result$="s" OR Result$="S"
GOSUB ServerSub
ENDIF
IF Result$="q" OR Result$="Q"
quit=1
ENDIF
WEND
// ------------------------------------------------------------- //
// --- CLIENT ---
// ------------------------------------------------------------- //
SUB ClientSub:
STDOUT "Running as client!\n"
LOCAL FileSize%
LOCAL ReceiveMessage$
LOCAL ReadFileSize=0 // How much data has been read so far
LOCAL FileHandle
FileHandle=GENFILE()
EstablishedConnection=SOCK_TCPCONNECT("127.0.0.1",11211) // Try to connect to server
IF EstablishedConnection > -1 // Success
Finished=0
WHILE Finished=0
DataLength=SOCK_RECV(EstablishedConnection,ReceiveMessage$,1024) // Try to read size of file to transmit
SELECT DataLength
CASE -1
DEBUG NETGETLASTERROR$()+"\n"
BREAK
CASE -2 // non blocking socket not would block
SLEEP 5
DEFAULT
Finished=1
ENDSELECT
WEND
Finished=0
FileSize%=ReceiveMessage$
OPENFILE(FileHandle, DestFile$, 0) // Open file for writing
SLEEP 200 // Let some data buffer
WHILE Finished=0
DataLength=SOCK_RECV(EstablishedConnection,ReceiveMessage$,1024) // Read 1024 bytes for the destination file
SELECT DataLength
CASE -1
DEBUG NETGETLASTERROR$()+"\n"
BREAK
CASE -2 // non blocking socket not would block
SLEEP 5
CASE >0
INC ReadFileSize, DataLength // Increase our count of how much has been transferred
WRITESTR FileHandle, ReceiveMessage$ // Write transferred data to the destination file
IF DataLength < 1024 THEN Finished=1 // When we read less than 1024 bytes, we've got all of them
STDOUT "\nRead " + ReadFileSize + " bytes of " + FileSize + " total."
ENDSELECT
WEND
STDOUT "\n\nOriginal size : "+FileSize
STDOUT "\nRead size : " + ReadFileSize
SOCK_CLOSE(EstablishedConnection)
ELSE
ErrorMsg$=NETGETLASTERROR$()
STDOUT "Error message = " + ErrorMsg$ + "\n"
ENDIF
CLOSEFILE FileHandle
ENDSUB // CLIENT
// ------------------------------------------------------------- //
// --- SERVER ---
// ------------------------------------------------------------- //
SUB ServerSub:
STDOUT "Server!\n"
LOCAL Done = 0
LOCAL ClientIPAddress%
LOCAL TransmitMessage$
LOCAL FileHandle
LOCAL FileName$
LOCAL DataByte%
LOCAL DataString$
LOCAL FileSize
LOCAL AmountSent=0
FileSize=GETFILESIZE(SourceFile$)
FileHandle=GENFILE()
FileOpened=OPENFILE(FileHandle,SourceFile$,1)
IF FileOpened=TRUE
DataString$=""
Sock=SOCK_TCPLISTEN(11211) // Listen for connections
STDOUT "Waiting for connection...\n"
WHILE Done=0
EstablishedConnection=SOCK_TCPACCEPT(Sock,ClientIPAddress) // Try to connect to a client
IF EstablishedConnection <> -1 // Create connection
STDOUT "Established connection...\n"
STDOUT "Sending filesize of " + FileSize + " to client\n"
SOCK_TCPSEND(EstablishedConnection,FileSize) // Send size to transfer to client
STDOUT "Sending file data...\n"
// Transmit file
WHILE ENDOFFILE(FileHandle) = FALSE
Done=1
READSTR FileHandle, DataString$, 1024 // Read 1024 bytes of the file
HasBeenSentFlag=0 // Reset flag to ensure files ending on 1024 byte boundaries don't get extra data sent
SOCK_TCPSEND(EstablishedConnection,DataString$) // Send 1024 bytes of data
INC AmountSent, 1024
STDOUT AmountSent + " bytes of " + FileSize + " bytes sent\n"
HasBeenSentFlag=1 // To ensure files ending on 1024 byte boundaries don't get an extra data packet sent
WEND
IF HasBeenSentFlag=0
SOCK_TCPSEND(EstablishedConnection,DataString$) // send last block of <1024 bytes of data
STDOUT "="
ENDIF
ENDIF
WEND
ENDIF
SOCK_CLOSE(Sock)
CLOSEFILE FileHandle
ENDSUB // SERVER
Testing it on a 300K txt file, it also seems to miss both the first and last 1Kb, but I'll figure that bug out later.
Ocean,
You have to have them both using the same port number - if they were on different ports they wouldn't be connecting to each other. It works as is for one way communication, trying to get two way working at the moment.
According to the TCP_ACCEPT man page :-
QuoteYou can now communicate with other processes by reading from or writing to this socket.
That implies two way communication on the one socket.
I have 2 way communication on the one socket, but issues with chr$(0) characters.
The following code transfers a source file on the server machine to a destination file on the client machine. Currently filenames and IP are hardcoded at the top of the file for testing purposes.
When run using localhost as both source and destination, binary files transfer correctly. When run on a LAN using 2 separate PCs, chr$(0) characters caused my ~300K test .exe to end up a ~7K file. ASCII files transfer correctly. Swapping the following lines around
INC DataString$, CHR$(0)
// INC DataString$, "a"
to do a string replacement of chr$(0) with "a" in the program result in the full size file being created (but obviously it's now corrupt).
Any ideas how I can get chr$(0) to transfer? I've tried reading/writing whole 1024byte strings, reading the file one character at a time (as per the code below) and reading the file as ubytes - none of the solutions work.
I've got no idea why it works when both server and client are the same machine either.
GLOBAL SourceFile$, DestFile$, ServerIP$, SocketNumber
SourceFile$="c:/glbasic/source.exe"
DestFile$="c:/temp/destination.exe"
ServerIP$="127.0.0.1"
SocketNumber=11211
// ------------------------------------------------------------- //
// --- MAIN ---
// ------------------------------------------------------------- //
SOCK_INIT()
LOCAL quit=0
WHILE quit=0
STDOUT "(C)lient / (S)erver / (Q)uit\n"
Result$ = INKEY$()
IF Result$="C" OR Result$="c"
GOSUB ClientSub
ENDIF
IF Result$="s" OR Result$="S"
GOSUB ServerSub
ENDIF
IF Result$="q" OR Result$="Q"
quit=1
ENDIF
WEND
// ------------------------------------------------------------- //
// --- CLIENT ---
// ------------------------------------------------------------- //
SUB ClientSub:
STDOUT "Running as client!\n"
LOCAL FileSize%
LOCAL ReceiveMessage$
LOCAL ReadFileSize=0 // How much data has been read so far
LOCAL FileHandle
FileHandle=GENFILE()
// Connect to server
EstablishedConnection=SOCK_TCPCONNECT(ServerIP$,SocketNumber) // Try to connect to server
IF EstablishedConnection > -1 // Success
Finished=0
WHILE Finished=0
DataLength=SOCK_RECV(EstablishedConnection,ReceiveMessage$,1024) // Try to read size of file to transmit
SELECT DataLength
CASE -1
DEBUG NETGETLASTERROR$()+"\n"
BREAK
CASE -2 // non blocking socket not would block
SLEEP 5
DEFAULT
Finished=1
ENDSELECT
WEND
// Send back confirmation of message receipt
FileSize%=ReceiveMessage$
STDOUT "\n\nReceived file size to transfer : "+FileSize
STDOUT "\nSending receipt of confirmation.\n"
CurrentPacketSizeSent=SOCK_TCPSEND(EstablishedConnection,"Received-proceed")
STDOUT "CurrentPacketSizeSent = " + CurrentPacketSizeSent + "\n"
// Receive data loop
STDOUT "Message Sent. Waiting for file data.\n" // Now receive data
OPENFILE(FileHandle, DestFile$, 0) // Open destination file for writing
Finished=0
WHILE Finished=0
DataLength=SOCK_RECV(EstablishedConnection,ReceiveMessage$,1024) // Read 1024 bytes for the destination file
SELECT DataLength
CASE -1
DEBUG NETGETLASTERROR$()+"\n"
BREAK
CASE -2 // non blocking socket not would block
SLEEP 5
CASE >0
INC ReadFileSize, DataLength // Increase our count of how much has been transferred
WRITESTR FileHandle, ReceiveMessage$ // Write transferred data to the destination file
IF DataLength < 1024 THEN Finished=1 // When we read less than 1024 bytes, we've got all of them
IF DataLength = 1 AND ReceiveMessage$="\0"
DEC ReadFileSize,1 // Ignore EOF marker
ENDIF
STDOUT "\nRead " + ReadFileSize + " bytes of " + FileSize + " total."
CASE 0
Finished=1
ENDSELECT
WEND
STDOUT "\n\nOriginal size : "+FileSize
STDOUT "\nRead size : " + ReadFileSize + "\n"
SOCK_CLOSE(EstablishedConnection) // Close socket
ELSE
ErrorMsg$=NETGETLASTERROR$()
STDOUT "Error message = " + ErrorMsg$ + "\n"
ENDIF
CLOSEFILE FileHandle // Close file
ENDSUB // CLIENT
// ------------------------------------------------------------- //
// --- SERVER ---
// ------------------------------------------------------------- //
SUB ServerSub:
STDOUT "Server!\n"
LOCAL Done = 0
LOCAL ClientIPAddress%
LOCAL TransmitMessage$
LOCAL FileHandle
LOCAL FileName$
LOCAL DataByte%
LOCAL DataString$
LOCAL FileSize
LOCAL TotalAmountSent=0,CurrentPacketSizeSent=0
LOCAL CurrentPacketCounter
LOCAL Loop
LOCAL CurrentByte$
LOCAL Diff
LOCAL ReceiveMessage$
FileSize=GETFILESIZE(SourceFile$)
FileHandle=GENFILE()
FileOpened=OPENFILE(FileHandle,SourceFile$,1)
DataString$=""
IF FileOpened=TRUE
// Wait for connection
Sock=SOCK_TCPLISTEN(SocketNumber) // Listen for connections
STDOUT "Waiting for connection...\n"
WHILE Done=0
EstablishedConnection=SOCK_TCPACCEPT(Sock,ClientIPAddress) // Establish client connection
IF EstablishedConnection > 0 // If connection was successful
Done=1
STDOUT "Established connection...\n"
STDOUT "Sending filesize of " + FileSize + " to client\n"
CurrentPacketSizeSent=SOCK_TCPSEND(EstablishedConnection,FileSize) // Send size to transfer to client
IF CurrentPacketSizeSent = -1 // ERROR!
STDOUT "ERROR! - " + NETGETLASTERROR$()+"\n"
DEBUG "ERROR! - " + NETGETLASTERROR$()+"\n"
BREAK
ENDIF
STDOUT "Waiting on confirmation of receipt.\n" // Wait for confirmation that client received the filesize
LOCAL ReceivedMessage=0
WHILE ReceivedMessage=0
ConfirmationReceived=SOCK_RECV(EstablishedConnection,ReceiveMessage$,1024)
SELECT ConfirmationReceived
CASE -1
DEBUG NETGETLASTERROR$()+"\n"
BREAK
CASE -2 // non blocking socket not would block
SLEEP 5
CASE > 0
ReceivedMessage=1
STDOUT "Message received -" + ReceiveMessage$ + ".\n"
ENDSELECT
WEND
// When the confirmation is received...
IF ReceiveMessage$="Received-proceed"
STDOUT "File size confirmed. Sending file data...\n"
// Transmit file loop
WHILE ENDOFFILE(FileHandle) = FALSE
DataString$=""
CurrentFilePos=FILEPOSITION(FileHandle)
IF FileSize-CurrentFilePos < 1024 // If there's less than 1024 bytes left in the file
Diff = FileSize - CurrentFilePos // We'll make a smaller "packet"
ELSE
Diff=1024
ENDIF
// Prepare 1024 byte data "packet"
FOR Loop=1 TO Diff
IF ENDOFFILE(FileHandle) = FALSE
READSTR FileHandle, CurrentByte$, 1 // Read a byte from the file
IF CurrentByte$="\0"
INC DataString$, CHR$(0)
// INC DataString$, "a"
ELSE
INC DataString$, CurrentByte$ // Assemble it into our 1024 byte transfer packet
ENDIF
ELSE
BREAK
ENDIF
NEXT
// Send data "packet" of 1024 bytes
CurrentPacketSizeSent=SOCK_TCPSEND(EstablishedConnection,DataString$) // Send 1024 bytes of data
IF CurrentPacketSizeSent = -1 // ERROR!
STDOUT "ERROR! - " + NETGETLASTERROR$()+"\n"
DEBUG "ERROR! - " + NETGETLASTERROR$()+"\n"
BREAK
ELSE
INC TotalAmountSent, CurrentPacketSizeSent // Keep track of amount of data transferred
STDOUT TotalAmountSent + " bytes of " + FileSize + " bytes sent\n"
ENDIF
ENDIF
// Send EOF to close remote connection
DataString$=CHR$(0)
CurrentPacketSizeSent=SOCK_TCPSEND(EstablishedConnection,DataString$) // Send EOF to close connection
WEND
ENDIF
WEND
ENDIF
SOCK_CLOSE(Sock) // Close socket
CLOSEFILE FileHandle // Close file
ENDSUB // SERVER
It looks like a bug if CHR$(0) isn't being transmitted. What you replace it with would depend on whether other value will be used -255 is usually a good value to use as an alternative.
When testing, testing network stuff on one machine is fraught with additional problems, not least the programs speed. I have also found that the programs tend to fight against each other (with the active program generally get both host and client messages) which cause additonal problems. If you can, try multiple machines.
Quote from: MrTAToad on 2009-Nov-06
It looks like a bug if CHR$(0) isn't being transmitted. What you replace it with would depend on whether other value will be used -255 is usually a good value to use as an alternative.
I'll post it as a bug in the bug forum and see what Gernot says.
I don't see how you could actually replace it with anything. In a binary file, each byte can only be one of 256 characters, and as far as I'm aware, every character is used (well, at least is likely to be used) in for example an exe file - therefore you can't replace it with anything. The only way I can see that you could replace it would be to encode every byte as a 2 byte sequence instead, but that means double the amounts of bytes transmitted for binary data. I don't want to get into more complex encoding schemes (or any encoding schemes actually!!!)
Other than the CHR$(0) issue, I found the program worked with itself without any issues - I agree that trying on multiple machines is wise though (which is when I found the "bug" :D )
Yes, it's a bug. True shame.
Is there any workaround? :giveup:
Possibly UUEncode the data into ASCII characters ?
Or encode numbers 0 to 255 as a 16-bit number ?
Quote from: MrTAToad on 2009-Nov-07
Possibly UUEncode the data into ASCII characters ?
Or encode numbers 0 to 255 as a 16-bit number ?
Good idea - that's pretty much what I put in my post in the bug forum. I was hoping for a GLBasic command solution though :whistle:
Bugfix in next update.