Tokenize a string with escaping: Difference between revisions

Content added Content deleted
(→‎{{header|Python}}: Added a functional draft (using reduce). Compatible with Python 3.)
(→‎Haskell Alternative: foldl -> foldr)
Line 1,095: Line 1,095:
===Alternative===
===Alternative===
This is essentially equivalent to the first (DFA) example, but, though possibly less elegant than the guard idiom, appears to be fractionally faster with larger (eg 180k) test strings.
This is essentially equivalent to the first (DFA) example, but, though possibly less elegant than the guard idiom, appears to be fractionally faster with larger (eg 180k) test strings.
<lang haskell>tokenize
<lang haskell>import Data.Bool (bool)

:: (Eq a, Foldable t)
=> a -> a -> t a -> [[a]]
tokenize :: Char -> Char -> String -> [String]
tokenize delim esc str = reverse $ reverse <$> (token : list)
tokenize delim esc str = reverse $ reverse <$> (token : list)
where
where
(token, list, _) =
(token, list, _) =
foldl
foldr
(\(aToken, aList, aEsc) x ->
(\x (aToken, aList, aEsc) ->
let literal = not aEsc
let literal = not aEsc
isEsc = literal && (x == esc)
isEsc = literal && (x == esc)
in if literal && (x == delim)
in bool
then ([], aToken : aList, isEsc)
(bool (x : aToken) aToken isEsc, aList, isEsc)
else ( if isEsc
([], aToken : aList, isEsc)
then aToken
(literal && x == delim))
else x : aToken
, aList
, isEsc))
([], [], False)
([], [], False)
str
(reverse str)


main :: IO ()
main :: IO ()
Retrieved from "https://rosettacode.org/wiki/Tokenize_a_string_with_escaping"