Author Topic: INI read... slow??? o.O  (Read 2864 times)

Offline Darmakwolf

  • Dr. Type
  • ****
  • Posts: 361
  • Aoooooooo - the werewolves of London!
    • View Profile
INI read... slow??? o.O
« on: 2012-Jul-24 »
I have a decent-size ini file for a map database. (366 kb, about 20,000 lines in it.) Just reading ONE value seems to lag my program for a good 5 seconds flat. Is there a better alternative to iniget$() ?

Offline Schranz0r

  • Premium User :)
  • Administrator
  • Prof. Inline
  • *******
  • Posts: 5111
  • O Rly?
    • View Profile
Re: INI read... slow??? o.O
« Reply #1 on: 2012-Jul-24 »
READ...

look in the Helpfile :)
I <3 DGArray's :D

PC:
AMD Ryzen 7 1700 @3.9GHz, 16GB HyperX Fury 3000MHz Ram, ASUS ROG GTX 1060 STRIX 6GB, Windows 10 Pro 64Bit, MSi Tomahawk B350 Mainboard

Offline Slydog

  • Prof. Inline
  • *****
  • Posts: 932
  • KodeSource
    • View Profile
    • KodeSource
Re: INI read... slow??? o.O
« Reply #2 on: 2012-Jul-24 »
You could try manually reading and parsing the data, using OPENFILE and READLINE.

Split the line into command / value by using SPLITSTR with a delimiter of "=".
(Or manually by using INSTR).
My current project (WIP) :: TwistedMaze <<  [Updated: 2015-11-25]

Offline Darmakwolf

  • Dr. Type
  • ****
  • Posts: 361
  • Aoooooooo - the werewolves of London!
    • View Profile
Re: INI read... slow??? o.O
« Reply #3 on: 2012-Jul-24 »
The problem with that is ... wouldn't I have to dump the entire file into memory in order to parse with INSTR or SPLITSTR? I really don't think that'd be too quick...

Offline Slydog

  • Prof. Inline
  • *****
  • Posts: 932
  • KodeSource
    • View Profile
    • KodeSource
Re: INI read... slow??? o.O
« Reply #4 on: 2012-Jul-24 »
I don't think so if you use READLINE.
It should only keep the current line in memory.
Then you process it, then read the next line.
My current project (WIP) :: TwistedMaze <<  [Updated: 2015-11-25]

Offline Wampus

  • Prof. Inline
  • *****
  • Posts: 1004
    • View Profile
Re: INI read... slow??? o.O
« Reply #5 on: 2012-Jul-24 »
If the whole map isn't loaded directly into memory (which should be fairly quick, especially with READIEEE) and strings aren't being read sequentially and but need to be random access then TBH it sounds like a carefully designed data structure would be better - something with a look-up table to keep track of where the strings are held and can deal with string splitting, a bit like file allocation and fragmentation.

Offline matchy

  • Prof. Inline
  • *****
  • Posts: 1545
    • View Profile
Re: INI read... slow??? o.O
« Reply #6 on: 2012-Jul-24 »
How about storing map layout data in a bitmap sprite?  :doubt:

Offline Darmakwolf

  • Dr. Type
  • ****
  • Posts: 361
  • Aoooooooo - the werewolves of London!
    • View Profile
SOLVED!
« Reply #7 on: 2012-Jul-24 »
Thanks to your suggestions - I re-wrote my map engine to use writestr. It's saved me a TON of loading time! Who knew iniput was that slow?! this is like a 2000% speed increase at least. Even loading a 300KB text file into memory AND parsing all of its contents takes ~1 second now. Thank you all :)

MrTAToad

  • Guest
Re: INI read... slow??? o.O
« Reply #8 on: 2012-Jul-26 »
INIGET and INIPUT should be fast...

Offline BdR

  • Dr. Type
  • ****
  • Posts: 303
    • View Profile
    • BdR Games
Re: INI read... slow??? o.O
« Reply #9 on: 2012-Jul-26 »
I guess INI files are meant to read and write a handfull of parameters for your app. Like sound on/off, last level played, lastplayer name and other game settings like that. So using an INI file to store such large maps seems like a mismatch.

Also, do you store the tiles something like this?
Code: (glbasic) [Select]
1=41
2=43
3=53
etc.
Or something like this
Code: (glbasic) [Select]
row1=13245463576799435
row2=24376516475168437
etc.
I think the first one would take more time than the second.

MrTAToad

  • Guest
Re: INI read... slow??? o.O
« Reply #10 on: 2012-Jul-26 »
Oh yes, INI files should not be used for anything other application settings - you would normally have map data as binary information so that it can't be easily modified, and is quick to load.

Offline BdR

  • Dr. Type
  • ****
  • Posts: 303
    • View Profile
    • BdR Games
Re: INI read... slow??? o.O
« Reply #11 on: 2012-Jul-26 »
Actually I also use INI files to store the level layouts in my puzle game. But those are very small tile maps, 8x8 tiles. So each level is a 128 character string which, for a setting, is kind of a long string but it's manageable and loading is fast enough.

I'd just use binary file for anything bigger than that, I mean like a 200x200 tiles Zelda-type map or Super Mario platform layout.

Offline hardyx

  • Community Developer
  • Prof. Inline
  • ******
  • Posts: 500
    • View Profile
Re: INI read... slow??? o.O
« Reply #12 on: 2012-Jul-26 »
Ini files are used for config settings, not for databases, because the reading/searching keys in a big file is slow.