Pfthb

I have a big string where I need to

1. Convert words starting with an upper-case to a lower-case word so that all words are lower case.



2. 
   

   Sort the amount of times a word is shown

   
   
   
   • A word in this sense is a sequence of characters without whitespaces
     or punctuation (!#= etc)
   
   



3. Sort from most frequent word shown and to less frequent word.

I've made a function to read a .txt file and turn it into a string.

But I'm not sure where to go from here and any kind of help with any of the bulletpoints would be greatly appreciated.

Let's go through this step by step then, creating a function for each bit:

Convert words starting with an upper-case to a lower-case word so that all words are lower case.

Split the string into a sequence of words:

let getWords (s: string) = 
 s.Split(' ')

Turns "hello world" into ["hello"; "world"]

Sort the amount of times a word is shown. A word in this sense is a sequence of characters without whitespaces or punctuation (!#= etc)

Part #1: Format a word in lower without punctuation:

let isNotPunctuation c = 
 not (Char.IsPunctuation(c))

let formatWord (s: string) = 
 let chars = 
 s.ToLowerInvariant()
 |> Seq.filter isNotPunctuation
 |> Seq.toArray

 new String(chars)

Turns "Hello!" into "hello".

Part #2: Group the list of words by the formatted version of it.

let groupWords (words: string seq) = 
 words
 |> Seq.groupBy formatWord

This returns a tuple, with the first part as the key (formatWord) the second part is a list of the words.

Turns ["hello"; "world"; "hello"] into

[("hello", ["hello"; "hello"]);
 ("world", ["world"])]

Sort from most frequent word shown and to less frequent word.

let sortWords group = 
 group
 |> Seq.sortByDescending (fun g -> Seq.length (snd g))

Sort the list descending (biggest first) by the length (count) of items in the second part - see the above representation.

Now we just need to clean up the output:

let output group =
 group
 |> Seq.map fst

This picks the first part of the tuple from the group:

Turns ("hello", ["hello"; "hello"]) into "hello".

Now we have all the functions, we can stick them together into one chain:

let s = "some long string with some repeated words again and some other words"

let finished = 
 s
 |> getWords
 |> groupWords
 |> sortWords
 |> output

printfn "%A" finished
//seq ["some"; "words"; "long"; "string"; ...]

Here's another way using Regex

open System.Text.RegularExpressions

let str = "Some (very) long string with some repeated words again, and some other words, and some punctuation too."

str 
|> (Regex @"W+").Split
|> Seq.choose(fun s -> if s = "" then None else Some (s.ToLower()))
|> Seq.countBy id
|> Seq.sortByDescending snd