While I was working on my project at Gobelins, I had to make a simple auto-completion system based on a French dictionary (336,532 words). Quite challenging but Philippe (a guy who is not too bad at programming, you know him?) was here to give me a hand.

We simply did this :
-load a *.txt containing the raw list
-split it on \n
-play with àäèéù….
-make an object containing 26 arrays

Once the file is loaded it takes 5 seconds to my 2-years old laptop to process the 300,000 words.
And Philip found an idea to save this processing time !
He wrote a simple AIR application that uses FileStream.writeObject to store the object into an external file !

The final application just have to load he file and use ByteArray.readObject to get the data with the good structure (and without any stack overflow :D)

For the second part (searching a word in the list) a simple for-each loop in the appropriate array did the job.

See you at Brighton (or before?)

(par respect pour tes lecteurs j’ai écrit en anglais, je crois pas que je le ferai deux fois :D)

you’re right about the special chars & the storage, I was thiking of special-char-replacing all the candidates and store the original strings somewhere.
then convert all the candidates into vectors of ints to perform both the Levenshtein & the ‘distance checks’.
this might be outrageously faster despite the pre process, should look into it :)

nice StringUtils class you have there btw :)

@philipp and you haven’t seen the worst names, sometimes I wonder about the parents’ sanity … btw it’s mostly girl’s name that are fucked up, I suspect some undergoing machism behind all that ( like “after all, a girl’s name doesn’t have to be serious if she’s to cook and look after the kids, let’s call her honey-cherry-love” ).

otherwise, I understand your point, it seems more ‘natural’ to have them sorted by length, it’s still possible to sort the results on length + alphabetically or to add their length to the score.

thanks to both of you for passing by :)

but just like peter mentioned, i tried some names and was wondering why i went from 10 matches to 1 to 5 to 1 to X. it’s a bit confusing – especially if you type sth like “ge” – didn’t know that this is an actual name ;) – and it only returns the “ge”. seemed broken at first sight – until i read the comments.

i prefer to add some potential matches at the end like so:

ge (as the exact match)
gees
geena
geete
geeske
geesje
geenia
geertrui

but may it’s just me or the way google has been conditioning me over the last years :D

keep up the geek stuff.
cheers phil

one thing that I kind wondered is that maybe it was going to be faster/easier to replace all the special chars of all the names before converting into an Array and keeping a copy of the original strings, you would use more memory but don’t know if it would affect that much… – maybe you don’t even need 2 arrays, just keep the original as a string and retrieve the original name based on the number of commas before it.

I know that the string methods (indexOf, etc…) are faster than RegExp but since you are doing on a loop maybe the RegExp approach would be faster. Here is a simple implementation of a special char replacement: StringUtils.replaceAccents.

Cheers!

@marpi true, I could have done that too :) I my case it was mandatory to have it all embedded.

@og2t I guess you could pile up a test where you write down the candidate in a textfield and store the textfield.textWidth property to alter the score :)

@cay I’m not at ease with regexp actually, it would surely save a lot of process time. I’ll give your option a thourough try on monday, thanks :)

@peter not working would mean ‘*no match found* BUT ;) in that case ‘ge’ is a perfect match and rule n°3 specifies that a perfect match returns a single string which is what occurs here :)

Cheers!