ARRAY SORT

   thinBasic web site
   What's new
   thinBasic help
   On Facebook
  
   Scripts repository
  
   Downloads
    - Stable version
  
   MIRROR SERVERs
    - Italy
    - US
  
   File repository

2 Spiders, 6 Guests, 1 User
Eros Olmi
Yahoo spider, Google spider

[Michael Clease]

I've suspect this for a while and thought I better prove it, the array sort doesnt sort correctly it doesnt take into account string length.

In my example the input should match the output but because string length is not checked it ignores the fact Word10 has an extra character and inserts it between Word1 and Word2 but it should appear at the end of the list.

Let me know what you think.

Mike

[Michael Hartlef]

Hi Michael,

as I agree with you that word10 should come after word2, its a normal behaviour when sorting strings. They get sorted after the ASC value of each character. If you have access to EXCEL, it does it the same.

Michael

[Eros Olmi]

Mike,

to let others understanding here the data (original and after sorting) that Mike is using:

Original	After sorting
Word1	Word1
Word2	Word10
Word3	Word2
Word4	Word3
Word5	Word4
Word6	Word5
Word7	Word6
Word8	Word7
Word9	Word8
Word10	Word9

ARRAY SORT uses ASCII sorting for strings and it compares equivalent ASCII code in the corresponding byte position
So "Word10" comes just after "Word1" and just before "Word2" because the 5th char is "1"

As Michael stated:

• Excel (any version up to 2007) does the same
• MSAccess (up to version 2007) does the same

But I do not want justify that current behave MUST be the only way.
For sure is not a bug so I will move this post in "Feature request" forum.
Instead it is and interesting observation that can bring to a new feature.
Considering how current ARRAY SORT works internally, honestly at the moment I do not know how to develop this new behave but worth to think about it.

Ciao
Eros

[Michael Clease]

But what kind of sorting is windows using for file details view its not what you are describing.

If I had the files

Fred-10.txt
Fred-11.txt
Fred-100.txt

the above is how they would be sorted and I would say that is a true sorting.

[Eros Olmi]

Of course it is a special sorting algorithm applied to file lists.
A standard LISTVIEW will always sort "File10" before "File2"
I think listview used by Explorer uses a callback function for getting the needed effect.

I'm not saying what you asked is not correct. I just said it is not the form generally used.

In any case I will think about it. I think I can intercept the sorting algorithm and do something.

Ciao
Eros

[Eros Olmi]

I Google around and as expected I found this is a common problem to many languages:
http://www.google.com/search?hl=en&source=hp&q=sorting+strings+with+numbers&aq=9&aqi=g10&oq=sorting+string

Someone call it "intuitive sorting"

A way could be to:

• get the array (only string arrays)
• create many subsets of it each composed by the strings having the same exact length
• sort each subset as usual
• recompose the whole array as a concatenation of sorted subsets

but .... what happen if file names are something like:
1.txt
1.1.txt
1.01.txt
1.001.txt
If sorted by size (like in file1.txt, file2.txt and file10.txt) you will get a completely different sorting because numerically file named "1.001.txt" should come first because it is the smallest number of the list.

Just brainstorming ...

[Michael Clease]

but .... what happen if file names are something like:
1.txt
1.1.txt
1.01.txt
1.001.txt
If sorted by size (like in file1.txt, file2.txt and file10.txt) you will get a completely different sorting because numerically file named "1.001.txt" should come first because it is the smallest number of the list.

Just brainstorming ...

The order you are showing is correct because it has nothing to do with what value but how many characters.

[Eros Olmi]

Do you think this kind of sorting can be applied only to files?

I mean, I was thinking about a special sorting algorithm working only on file names because it is not the string length that makes the different but the file name without extension.
In this can I can scan the string array, remove the file extension and work on file name length.
Than return an "intuitive" sorted array.

[Eros Olmi]

I have something to test for you Mike.

I've changed ARRAY SORT syntax to the following:

Code

ARRAY SORT ArrayVariable([StartIndex]) [FOR nElements] [,{ASCEND | DESCEND}] [, ASFILES]

so you can call it something like

Code

Array Sort sArray, AsFiles
Array Sort sArray, Ascend, AsFiles
Array Sort sArray, Descend, AsFiles
 

Here it is the way clause ASFILE works:

• right search into array element for file extension and get it off
• in the remaining part it right search for any numeric char in the range (0123456789.) and get it off
• at this point we have 3 parts:
  
  • (A) left part of the string
  • (B) the numeric part (it can be missed)
  • (C) the extension of the file (it can be missed)
• now the string is filled with NULLs ( CHR$(0) ) up to 32 bytes in the following way: (A) & NULLs & (B) & (C)
  The NULLs part is variable in length depending on how many numeric chars (B part) are present
  In few words the numeric part of the file name is normalized with NULLs up to a certain size in order to have all file names with the same size.
  For example:
  file1.txt   is transformed into file______1.txt
  file10.txt  is transformed into file_____10.txt
  file2.txt   is transformed into file______2.txt
  with _ be $NUL chars
• array is sorted
• NULLs are removed
• sorted array is returned

Let me know if it works or there are cases where it creates problems.
Maybe I will change clause ASFILES to something ASNATURAL but for this tests just use ASFILES

Ciao
Eros

ATTENTION !!!! Attached thinCore.dll works only if you have thinBasic 1.7.10.1 installed otherwise do not install !!!!!!!



Updated 2009.02.08

[Michael Clease]

Thanks for all your hard work but it managed to break it

If you use this string in my script with the added "asfiles" it blanks the array

 Txt = "1,2,3,11,111,1111,11111,2222,22"

For syntax can I suggest
 

Code

ARRAY SORT ArrayVariable([StartIndex]) [FOR nElements] [,{ASCEND | DESCEND}] [, Full ]
 

the Full option being either %True or %False the default being %False because it will be faster I am guessing.

[Eros Olmi]

Mike,

ok, thanks.
In any case I think I'm close to a solution.
I will go on this evening when at home.

Eros

[Eros Olmi]

Mike,

I've updated thinCore.dll in above post here just above
http://community.thinbasic.com/index.php?topic=3196.msg23973#msg23973

I hope I've fixed it.
Let me know if you find other cases where it doesn't work.

Eros

[Michael Clease]

Input Text = "AAA,AAB,AAC,AAAA,AABA,ABBA"

Code

Text Before Sort             Text After Sort
 
AAA                          AAA
AAB                          AAAA
AAC                          AAB
AAAA                         AABA
AABA                         AAC
ABBA                         ABBA

I would still say its not working but do we have the same goal?  in this example I would expect the output to match the input.

[Eros Olmi]

Updated again above thinCore.zip that should take case also of latest example.

Yes we are going same direction but that direction seems more complex than expected.
Also I'm not sure I've used the best possible algorithm.
There are a lot of limitations in current algorithm regarding size of strings that will be possible to sort in this way.
I will talk more in future posts as soon as I will have some spare time.

Let me know if with latest Core there are still uncovered situations.

Ciao
Eros

[Michael Clease]

Thank you as always it seem to be working but I push it around at a more civlilised hour, the only other thing that is open to interpretation are the characters  !"£$%^&*()_=+ ... etc but it seems really good to me.

Thanks again.

Mike

[Tags:]

Jump to: Please select a destination: ----------------------------- General community ----------------------------- => Shout Box Area => General ===> Events ===> PCOPY! ===> Development ===> 3rd party library ===> Science ===> Technology ===> Search engines ===> Software discussion ===> Gaming ===> Other programming languages =====> C / C++ =====> Free Basic =====> Free Pascal =====> Power Basic =======> Power Basic tools and resources =======> Power Basic specific boards =========> Power Basic OpenGL =========> Power Basic and DBMS =========> Power Basic COM =======> Power Basic sources =====> Pure Basic =====> ScriptBasic =====> Scripting ===> Security => Announcements ===> thinBasic Beta testing ===> Server, web site and Forum updates ===> vBulletin new forum updates => ThinBasic Journal ===> Suggest new articles => thinStudio ===> thinGaming Scoring web site ===> CM contest 2009 =====> Media =====> Code ----------------------------- User projects ----------------------------- => User files and/or user projects ===> T.A.B. (ThinBasic Adventure Builder) ----------------------------- thinBasic (the language) ----------------------------- => General ===> Polls ===> Installation ===> Help Material =====> Fixed or cleared errors in help material ===> YouTube ===> Challenge ===> thinBasic development blog ===> thinBasic vaporware :D ===> thinBasic where ... => Modules specific issues ===> Suggest new module ===> BIFF ===> CGI and FastCGI ===> COM ===> COMM (Serial communication module) ===> Console =====> Console: source code examples for beginners =====> Console: advanced source code examples ===> Core module ===> Dictionary module ===> DT (Date module) ===> Eval ===> EXE ===> File ===> iComplex ===> INet ===> INI ===> Irrlicht ===> LAN ===> ODBC ===> OS ===> Statistics Module ===> TBASS ===> Tcp/Udp module ===> TImage ===> Tokenizer ===> UI (User Interface) module ===> VBRegExp ===> Win API interface ===> WMI => Sources, Templates, Code Snipets, Tips & Tricks, Do you know ... ===> Do you know ... ===> thinBasic tutorials =====> General purpose scripts =====> Execution speed tests =====> Files and directories handling =====> Internet related scripts =====> Math scripts =======> Eval =====> Tips & Tricks =====> User Interface scripts ===> Samples for help file ===> Templates ===> Real world situations and solutions using thinBasic =====> Project: 3D Warehouse ===> Math: all about => Suggest new features ===> New features: Developed or solved => Bug report ===> Fixed or cleared bug reports ===> thinBasic Beta testing suggestions or bug reports ----------------------------- thinAir (the IDE) ----------------------------- => thinAir General ===> thinAir Tips and Tricks ===> thinAir CodeTips/CodeSense => thinAir Suggest new features ===> thinAir new features developed => thinAir Bug report ===> thinAir Fixed or Cleared Bug report => Alternative IDEs ----------------------------- thinTools ----------------------------- => thinSVN => thinDebug => thinBundle ===> thinBundle suggest new features ===> thinBundle bugs report => thinBasic SDK ===> Basic language SDK development =====> Module SDK (Power Basic version) =====> Module SDK (Freebasic version) ===> Turbo Delphi language SDK development ===> C/C++ language SDK development =====> Module SDK (Microsoft C /C++) =======> MSVC++ 6.0 =======> MSVC .NET =====> Module SDK (Borland C /C++) =====> Module SDK (GCC C /C++) ===> Other languages SDK development =====> Module SDK (MASM32 assembler) => User tools => 3rd party tools ----------------------------- User modules (the user developed modules expansions) ----------------------------- => TBGL module by Petr Schreiber ===> TBGL General ===> TBGL Bonus Pack ===> TBGL Bonus Pack Pro ===> TBGL Suggest new features ===> TBGL Bug Report ===> TBGL Scripts and Projects =====> TBGL Random Trees =====> TBGL Robot duel =====> TBGL Labyrinth =====> Top Down 3D =======> Game general =======> Game story and Character Development =======> Game executable =======> Game sources =======> Media material =======> Bugs handling =====> TBGL ScreenSavers =====> TBGL Gui projects =====> TBGL nmg_tankwars =====> TBGL Arkanoid ===> TBGL Modelling, 3D, other SW integration ===> M15 file format ===> Resources ===> TBGL Tutorials =====> ThinBASIC programming in OpenGL/TBGL ===> TBGL Font Creator => TBDI module. thinBasic Direct Input integration by MikeHart ===> TBDI Bug report => TBEM module - thinBasic Event Manager module => O2 JIT Compiler / Assembler ===> O2h Compiler ===> Machine Code =====> Machine Code references =====> Machine Code: script sources and examples ===> Programs ===> Legacy => Experimental modules or library interface ===> AStar - (A Star) path-finding algorithm ===> ODE - Open Dynamics Engine ===> OpenCL ===> tb_graphx ===> TBCDL Collision Dection Library ===> TBTIL Texture Image Library ----------------------------- Utilities working with thinBasic ----------------------------- => Tray Agent