Strip control codes and extended characters from a string: Difference between revisions

{{task}}

 

 

 

 

On a non-ASCII based system, we consider characters that do not have a corresponding glyph on the ASCII table (within the ASCII range of 32 to 126 decimal) to be an extended character for the purpose of this task.

<br><br>

 

=={{header|Ada}}==

 

 

procedure Strip_ASCII is

Put_Line("Neither_Extended:", Filter(Full, Above => Character'Last)); -- defaults for From and To

end Strip_ASCII;

 

Output:

 

=={{header|ALGOL 68}}==

# assums ASCII is the character set #

PROC strip characters = ( STRING text, BOOL strip extended )STRING:

STRING t = REPR 2 + "abc" + REPR 10 + REPR 160 + "def~" + REPR 127 + REPR 10 + REPR 150 + REPR 152 + "!";

print( ( "<<" + t + ">> - without control characters: <<" + strip characters( t, FALSE ) + ">>", newline ) );

{{out}}

<pre>

<<�abc

ûÿ!>> - without control characters: <<abcádef~ûÿ!>>

<<�abc

ûÿ!>> - without control or extended characters: <<abcdef~!>>

</pre>

 

=={{header|AutoHotkey}}==

{{trans|Python}}

Loop Parse, x

if Asc(A_LoopField) > 31 and Asc(A_LoopField) < 128

return r

}

 

=={{header|AWK}}==

# syntax: GAWK -f STRIP_CONTROL_CODES_AND_EXTENDED_CHARACTERS.AWK

BEGIN {

exit(0)

}

<p>output:</p>

<pre>

control and extended stripped: abz (length 3)

</pre>

=={{header|BASIC}}==

{{works with|QBasic}}

While DOS does support ''some'' extended characters, they aren't entirely standardized, and shouldn't be relied upon.

 

DECLARE FUNCTION strip2$ (what AS STRING)

 

NEXT

strip2$ = outP

 

Output:

 

=={{header|BBC BASIC}}==

PRINT "Original ISO-8859-1 string: " test$ " (length " ; LEN(test$) ")"

test$ = FNstripcontrol(test$)

ENDIF

ENDWHILE

Output:

<pre>

</pre>

 

=={{header|Bracmat}}==

characters and other ilk.\L\D§►↔◄

Rødgrød med fløde"

)

)

Output:

<pre>Control characters stripped:

 

=={{header|C}}==

#include <stdlib.h>

 

 

return 0;

!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~

 

=={{header|C sharp|C#}}==

Uses the test string from REXX.

using System;

using System.Collections.Generic;

}

}

Output:

<pre>

Stripped of extended: string of , may include control characters and other ilk.

</pre>

 

=={{header|Clojure}}==

(def range-of-chars (apply str (map char (range 256))))

 

 

; filter to return String of characters that are between 32 - 126:

 

=={{header|Common Lisp}}==

 

 

 

=={{header|D}}==

 

S stripChars(S)(S s, bool function(dchar) pure nothrow mustStrip)

writeln(s.stripChars( c => isControl(c) || c == '\u007F' ));

writeln(s.stripChars( c => isControl(c) || c >= '\u007F' ));

{{out}}

abcédef

abcdef</pre>

 

=={{header|Erlang}}==

Exported functions to be used by [[Update_a_configuration_file]]

-module( strip_control_codes ).

 

String_without_cc_nor_ec = lists:filter( fun is_not_control_code_nor_extended_character/1, String ),

io:fwrite( "String without control codes nor extended characters (~p characters): ~s~n", [erlang:length(String_without_cc_nor_ec), String_without_cc_nor_ec] ).

{{out}}

<pre>

41> strip_control_codes:task().

String (256 characters): ^@^A^B^C^D^E^F^G^H

String without control codes nor extended characters (95 characters): !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~

</pre>

=={{header|F Sharp|F#}}==

Uses test string from REXX.

open System

 

printfn "Stripped of extended: %s" (stripExtended test)

0//main must return integer, much like in C/C++

Output:

<pre>

Stripped of extended: string of , may include control characters and other ilk.

</pre>

 

=={{header|Factor}}==

 

: strip-control-codes ( str -- str' ) [ control? not ] filter ;

 

: strip-control-codes-and-extended ( str -- str' )

 

=={{header|Forth}}==

over + over swap over ( buf dst limit src )

do

then

loop

 

=={{header|Fortran}}==

implicit none

 

write (*,*) strip(string,not_extended)

end program test

 

=={{header|FreeBASIC}}==

 

Function stripControlChars(s As Const String) As String

Print

Print "Press any key to quit"

 

{{out}}

Ext chars stripped Length => 25

Both sets stripped Length => 19

</pre>

 

=={{header|Gambas}}==

'''[https://gambas-playground.proko.eu/?gist=19db658a6c44cfb1f6f887ff53e549bb Click this link to run this code]'''

Dim sString As String = "The\t \equick\n \fbrownfox \vcost £125.00 or €145.00 or $160.00 \bto \ncapture ©®"

Dim sStd, sExtend As String

Return sResult

 

Output:

<pre>

=={{header|Go}}==

Go works for ASCII and non-ASCII systems. The first pair of functions below interpret strings as byte strings, presumably useful for strings consisting of ASCII and 8-bit extended ASCII data. The second pair of functions interpret strings as UTF-8.

 

import (

fmt.Println("\nas decomposed and stripped Unicode:")

fmt.Println(stripCtlAndExtFromUnicode(src))

Output: (varies with display configuration)

<pre>

source text:

déjà vu

as⃝df̅

 

 

=={{header|Groovy}}==

Test:

def textMinusControl = text.findAll { int v = (char)it; v > 31 && v != 127 }.join('')

def textMinusControlAndExtended = textMinusControl.findAll {((char)it) < 128 }.join('')

 

assert stripControl(text) == textMinusControl

 

=={{header|Haskell}}==

 

strip = filter (liftA2 (&&) (> 31) (< 126) . fromEnum)

 

main :: IO ()

{{Out}}

 

=={{header|Icon}} and {{header|Unicon}}==

We'll use ''deletec'' to remove unwanted characters (2nd argument) from a string (1st argument). The procedure below coerces types back and forth between string and cset. The character set of unwanted characters is the difference of all ASCII characters and the ASCII characters from 33 to 126.

write(image(deletec(&ascii,&ascii--(&ascii)[33:127])))

end

link strings

 

{{libheader|Icon Programming Library}}

 

The IPL procedure ''deletec'' is equivalent to this:

result := ""

s ? {

return result ||:= tab(0)

}

 

 

=={{header|J}}==

'''Solution:'''

'''Usage:'''

#mystring NB. length of string

256

95

stripControlExtCodes myunicodestring

 

[[Category:String manipulation]]

 

=={{header|JavaScript}}==

===ES 5===

 

 

// s -> s

return strip(strTest);

 

 

{{Out}}

 

 

=={{header|jq}}==

{{works with|jq|1.4}}

explode | map(select(. > 31 and . != 127)) | implode;

 

def strip_extended_characters:

 

'''Example''':

 

"string | strip_control_codes\n => \(string | strip_control_codes)",

{{out}}

string | strip_control_codes

=> string of ☺☻♥♦⌂, may include control characters such as null() and other ilk.§►↔◄Rødgrød med fløde

string | strip_extended_characters

 

=={{header|Julia}}==

stripc0{T<:String}(a::T) = replace(a, r"[\x00-\x1f\x7f]", "")

stripc0x{T<:String}(a::T) = replace(a, r"[^\x20-\x7e]", "")

println("\nWith C0 control characters removed:\n ", stripc0(a))

println("\nWith C0 and extended characters removed:\n ", stripc0x(a))

 

{{out}}

 

=={{header|Kotlin}}==

 

fun String.strip(extendedChars: Boolean = false): String {

val u = s.strip(true)

println("String = $u Length = ${u.length}")

 

{{out}}

<pre>

Originally:

 

After stripping control characters:

String = 123abcDEF+-*/ Length = 13

</pre>

 

=={{header|Liberty BASIC}}==

all$ =""

for i =0 to 255

extendedStripped$ =r$

end function

=={{header|Lua}}==

local s = ""

for i in str:gmatch( "%C+" ) do

 

print( Strip_Control_Codes(q) )

!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~</pre>

 

MemberQ[CharacterRange["!","~"]~Join~Characters[FromCharacterCode[Range[128,255]]],#]&]]

 

stripCtrlExt[x_]:=StringJoin[Select[Characters[x],

 

Test:

\.15\.16\.17\.18\.19\.1a\[RawEscape]\.1c\.1d\.1e\.1f !"#$%&'()*+,-./

0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]

\[PlusMinus]\.b2\.b3\.b4\[Micro]\[Paragraph]\[CenterDot]¸¹º»¼½¾¿

ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ*ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö/øùúûüýþÿ

stripCtrl[CompleteSet]

->!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]

\[PlusMinus]\.b2\.b3\.b4\[Micro]\[Paragraph]\[CenterDot]

¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ*ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö

->!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]

^_`abcdefghijklmnopqrstuvwxyz{|}~</pre>

 

=={{header|MATLAB}} / {{header|Octave}}==

 

str = str(31<str & str<127);

 

=={{header|Nim}}==

result = ""

for c in str:

result.add c

 

Output:

 

=={{header|OCaml}}==

 

let is_extended_char c =

let strip f str =

let len = String.length str in

let rec aux i j =

if i >= len

else if f str.[i]

then aux (succ i) j

in

aux 0 0

let () =

Random.self_init ();

in

print_endline (strip is_control_code s);

 

=={{header|Pascal}}==

{{works with|Free_Pascal}}

 

function Strip (s: string; control, extended: boolean): string;

writeln ('No extnd: ', Strip(test, false, true));

writeln ('ASCII: ', Strip(test, true, true));

Output:

<pre>% ./StripCharacters

=={{header|Peloton}}==

Peloton has a native instruction for removing control codes from a string, SAL, the Low ASCII Strip. From the manual:

- - - - - -

 

Assign infix <@ LETVARSALVAR>j|i</@> <@ SAYVAR>j</@>

Assign prepend <@ LETSALVARVAR>k|i</@> <@ SAYVAR>k</@>

 

Peloton also has SAH, High ASCII Strip. Again, from the manual:

Strip high ANSI <@ SAYSAHVAR>i</@>

Assign infix <@ LETVARSAHVAR>j|i</@> <@ SAYVAR>j</@>

Assign prepend <@ LETSAHVARVAR>k|i</@> <@ SAYVAR>k</@>

 

=={{header|Perl}}==

use strict ;

 

print "\nWithout extended: " ;

print join( '' , map { chr( $_ ) } @noextended ) ;

Output:

<PRE>before sanitation : �L08&YH�O��n)�:���O�G$���.���"zO���Q�?��

Without extended: L08&YHOn):OG$."zOQ?

</PRE>

 

=={{header|Phix}}==

While you can delete a character from a string using say s[i..i] = "", the fastest and easiest way is always just

to build a new one character-by-character.<br>

{{out}}

<pre>

No Control Chars: " abc+®def", Length:9

" and no Extended: " abcdef", Length:7

</pre>

 

=={{header|PicoLisp}}==

Control characters in strings are written with a hat (^) in PicoLisp. ^? is the DEL character.

(pack

(filter

(filter

'((C) (> "^?" C "^_"))

Test:

<pre>: (char "^?")

 

=={{header|Pike}}==

> (string)((array)input-enumerate(32)-enumerate(255-126,1,127));

 

=={{header|PL/I}}==

stripper: proc options (main);

declare s character (100) varying;

 

end stripper;

Output:

<pre>

 

=={{header|PowerShell}}==

function Remove-Character

{

}

}

 

$test = "$([char]9)Français."

 

"Extended characters stripped : `"$($test | Remove-Character -Extended)`""

"Control & extended stripped : `"$($test | Remove-Character)`""

{{Out}}

<pre>

Control & extended stripped : "Franais."

</pre>

"Français", "Čeština" | Remove-Character -Extended

{{Out}}

<pre>

 

=={{header|PureBasic}}==

Protected i, *ptrChar.Character, length = Len(source), result.s

*ptrChar = @source

Print(#CRLF$ + #CRLF$ + "Press ENTER to exit"): Input()

CloseConsole()

Sample output:

<pre>»╫=┐C─≡G(═ç╤â√╝÷╔¬ÿ▌x  è4∞|)ï└⌐ƒ9²òτ┌ºáj)▓<~-vPÿφQ╨ù¿╖îFh"[ü╗dÉ₧q#óé├p╫■

 

=={{header|Python}}==

 

 

=={{header|Racket}}==

 

#lang racket

;; Works on both strings (Unicode) and byte strings (raw/ASCII)

(define (strip-controls-and-extended str)

(regexp-replace* #rx"[^\040-\176]+" str ""))

 

=={{header|REXX}}==

===idiomatic version===

This REXX version processes each character in an idiomatic way &nbsp; (if it's a wanted character, then keep it).

z= 'string of ☺☻♥♦⌂, may include control characters and other ♫☼§►↔◄░▒▓█┌┴┐±÷²¬└┬┘ilk.'

@=' !"#$%&''()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~'

 

say 'old = »»»'z"«««" /*add ««fence»» before & after old text*/

{{out|output}}

<pre>

 

Because there are &nbsp; (or should be) &nbsp; fewer unwanted characters than wanted characters, this version is faster.

x= 'string of ☺☻♥♦⌂, may include control characters and other ♫☼§►↔◄░▒▓█┌┴┐±÷²¬└┬┘ilk.'

@=' !"#$%&''()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghij' || ,

 

say 'old = »»»' || x || "«««" /*add ««fence»» before & after old text*/

{{out|output|text=&nbsp; is identical to the 1^st REXX version.}} <br><br>

 

=={{header|Ring}}==

s = char(31) + "abc" + char(13) + "def" + char(11) + "ghi" + char(10)

see strip(s) + nl

next

return strip

 

=={{header|Ruby}}==

def strip_control_characters()

chars.each_with_object("") do |char, str|

p s = "\ba\x00b\n\rc\fd\xc3\x7ffoo"

p s.strip_control_characters

 

{{out}}

 

=={{header|Run BASIC}}==

print strip$(s$)

 

end if

next i

<pre>

input : chr$(31)+"abc"+chr$(13)+"def"+chr$(11)+"ghi"+chr$(10)

output : abcdefghi</pre>

 

=={{header|Scala}}==

===ASCII: Using StringOps Class===

val extendedCode : (Char) => Boolean = (c:Char) => (c <= 32 || c > 127)

 

 

println( "ctrl and extended filtered out: \n\n" +

{{out}}

<pre>ctrl filtered out:

 

===Unicode: Using Regular Expressions===

// A Unicode test string

//

val htmlNoExtCode = for( i <- sNoExtCode.indices ) yield

"&#" + sNoExtCode(i).toInt + ";" + (if( (i+1) % 10 == 0 ) "\n" else "")

{{out}}

<pre>ctrl filtered out:

&#32;&#68;&#113;&#49;&#91;&#58;&#51;&#83;&#80;&#77;&#110;&#67;&#90;&#60;&#74;&#43;&#102;&#117;&#34;&#93;&#109;&#92;&#50;&#54;&#124;&#106;&#85;&#64;&#101;&#96;&#78;&#63;&#95;&#39;&#75;&#126;&#112;&#115;&#46;&#98;&#105;&#72;&#55;&#62;&#122;&#65;&#88;&#86;&#70;&#53;

&#61;&#79;&#103;&#66;&#104;&#89;&#71;&#99;&#45;&#52;&#41;&#69;&#47;&#42;&#97;&#44;&#37;&#119;&#84;&#76;&#111;&#82;&#38;&#87;&#123;&#107;&#100;&#125;&#56;&#108;&#94;&#59;&#48;&#35;&#40;&#33;&#116;&#114;&#118;&#73;&#120;&#36;&#81;&#121;&#57;</pre>

 

=={{header|Seed7}}==

Unicode characters with UTF-8 encoding to the console.

 

include "utf8.s7i";

 

writeln("Stripped of control codes and extended characters:");

writeln(stripControlAndExtended(src));

 

Output:

source text:

déjà vu

as⃝df̅

Stripped of control codes:

Stripped of control codes and extended characters:

dj vu !~asdf

 

=={{header|Sidef}}==

 

var letters = str.chars.map{.ord}

 

var noextended = nocontrols.grep{ _ < 127 }

{{out}}

<pre>

"abcdfoo"

</pre>

 

=={{header|Tcl}}==

regsub -all {[\u0000-\u001f\u007f]+} $str ""

}

proc stripCC str {

regsub -all {[^\u0020-\u007e]+} $str ""

 

=={{header|TI-83 BASIC}}==

The following "normal characters" do exist, but can't be typed on the calculator and a hex editor must be used to enter them:

 

 

The double quote character (ASCII decimal 34) can be entered, but cannot be escaped and thus cannot be stored to strings without the use of hex editors. The following program will remove double quotes from the input string if they were hacked in simply because having one stored to the "check" string is syntactically invalid.

So, in sum, you have to hack the calculator to enter in this program, but once it's entered you can transfer it to unhacked calculators and it will work.

 

:Input ">",Str1

:":"+Str1+":"→Str1

:End

:sub(Str1,2,length(Str1)-1)→Str1

 

=={{header|TXR}}==

{{trans|Racket}}

 

(regsub #/[\x0-\x1F\x7F]+/ "" str))

 

(defun strip-controls-and-extended (str)

 

=={{header|VBScript}}==

Derived from the BASIC version.

Function StripCtrlCodes(s)

tmp = ""

End Function

 

WScript.StdOut.WriteLine

WScript.StdOut.WriteLine

 

{{Out}}

<pre>

</pre>

 

=={{header|XPL0}}==

string 0; \use zero-terminated string convention

 

Strip(String, true);

Text(0, String); CrLf(0);

 

Output:

=={{header|zkl}}==

ASCII

var extdChars=[127..255].pump(String,"toChar");

 

(test-extdChars).println("<< no extended chars");

(test-extdChars-ctlCodes).println("<< text");

{{out}}

<pre>