Read a file character by character/UTF8: Difference between revisions

 

=={{header|Action!}}==

Proc Main()

Close(1)

 

=={{header|AutoHotkey}}==

{{works with|AutoHotkey 1.1}}

while !File.AtEOF

 

=={{header|C}}==

#include <wchar.h>

#include <stdlib.h>

 

return EXIT_SUCCESS;

 

=={{header|C++}}==

#include <fstream>

#include <iostream>

return EXIT_SUCCESS;

}

 

=={{header|C sharp|C#}}==

using System.IO;

using System.Text;

}

}

 

=={{header|Common Lisp}}==

{{works with|CLISP}}{{works with|Clozure CL}}{{works with|CMUCL}}{{works with|ECL (Lisp)}}{{works with|SBCL}}{{works with|ABCL}}

 

#+clisp (import 'charset:utf-8 'keyword)

 

(loop for c = (read-char s nil)

while c

 

=={{header|Crystal}}==

The encoding is UTF-8 by default, but it can be explicitly specified.

 

file.each_char { |c| p c }

 

or

 

while c = file.read_char

p c

end

=={{header|Delphi}}==

{{libheader| System.SysUtils}}

{{libheader| System.Classes}}

{{Trans|C#}}

program Read_a_file_character_by_character_UTF8;

 

end;

readln;

 

=={{header|Déjà Vu}}==

 

local (read-utf8-char) file tmp:

!read-byte file

!close file

return

 

=={{header|Factor}}==

IN: rosetta-code.read-one

 

"input.txt" utf8 [

[ read1 dup ] [ 1string write ] while drop

 

 

=={{header|FreeBASIC}}==

f = Freefile

 

Wend

Close #f

 

=={{header|FunL}}==

 

r = InputStreamReader( FileInputStream('input.txt'), 'UTF-8' )

print( chr(ch) )

 

=={{header|Go}}==

 

import (

fmt.Printf("%c", r)

}

 

=={{header|Haskell}}==

{{Works with|GHC|7.8.3}}

 

 

{- The procedure to read a UTF-8 character is just:

xs -> forM_ xs $ \name -> do

putStrLn name

{{out}}

<pre>

First, we know that the first 8-bit value in a utf-8 sequence tells us the length of the sequence needed to represent that character. Specifically: we can convert that value to binary, and count the number of leading 1s to find the length of the character (except the length is always at least 1 character long).

 

 

So now, we can use indexed file read to read a utf-8 character starting at a specific file index. What we do is read the first octet and then read as many additional characters as we need based on whatever we started with. If that's not possible, we will return EOF:

 

try.

octet0=. 1!:11 y;x,1

'EOF'

end.

 

The length of the result tells us what to add to the file index to find the next available file index for reading.

 

=={{header|Java}}==

import java.io.IOException;

import java.nio.charset.StandardCharsets;

 

 

}

 

=={{header|jq}}==

jq being stream-oriented, it makes sense to define `readc` so that it emits a stream of the UTF-8 characters in the input:

 

Example:

echo '过活' | jq -Rn 'include "readc"; readc'

"过"

"活"

 

=={{header|Julia}}==

The built-in <code>read(stream, Char)</code> function reads a single UTF8-encoded character from a given stream.

 

while !eof(f)

c = read(f, Char)

println(c)

end

 

=={{header|Kotlin}}==

 

import java.io.File

}

}

 

=={{header|Lua}}==

{{works with|Lua|5.3}}

-- Return whether the given string is a single ASCII character.

function is_ascii (str)

end

end

{{out}}

𝄞 A ö Ж € 𝄞 Ε λ λ η ν ι κ ά y ä ® € 成 长 汉

from revision 27, version 9.3, of M2000 Environment, Chinese 长 letter displayed in console (as displayed in editor)

 

Module checkit {

\\ prepare a file

}

checkit

 

using document as final$

 

Module checkit {

\\ prepare a file

checkit

 

 

=={{header|Mathematica}}/{{header|Wolfram Language}}==

 

=={{header|NetRexx}}==

 

:The file <tt>data/utf8-001.txt</tt> it a UTF-8 encoded text file containing the following:&nbsp;&#x79;&#xE4;&#xAE;&#x20AC;&#x1D11E;&#x1D122;&#x31;&#x32;.

options replace format comments java crossref symbols nobinary

numeric digits 20

say

return

{{out}}

<pre>

As in fact the file would be read line by line, even if the characters are actually yielded one by one, it may be considered as cheating. So, we provide a function and an iterator which read bytes one by one.

 

 

proc readUtf8(f: File): string =

res.add f.readChar()

if res.validateUtf8() == -1: break

 

=={{header|Pascal}}==

program ReadFileByChar;

var

Close(OutputFile)

end.

=={{header|Perl}}==

 

open my $fh, "<:encoding(UTF-8)", "input.txt" or die "$!\n";

}

 

 

If the contents of the ''input.txt'' file are <code>aă€⼥</code> then the output would be:

could easily add this to that file permanently, and document/autoinclude it properly.

 

<span style="color: #008080;">constant</span> <span style="color: #000000;">INVALID_UTF8</span> <span style="color: #0000FF;">=</span> <span style="color: #000000;">#FFFD</span>

<span style="color: #008080;">return</span> <span style="color: #000000;">res</span>

<span style="color: #008080;">end</span> <span style="color: #008080;">function</span>

 

Test code:

 

<span style="color: #000080;font-style:italic;">--string utf8 = "aă€⼥" -- (same results as next)</span>

<span style="color: #004080;">string</span> <span style="color: #000000;">utf8</span> <span style="color: #0000FF;">=</span> <span style="color: #7060A8;">utf32_to_utf8</span><span style="color: #0000FF;">({</span><span style="color: #000000;">#0061</span><span style="color: #0000FF;">,</span><span style="color: #000000;">#0103</span><span style="color: #0000FF;">,</span><span style="color: #000000;">#20ac</span><span style="color: #0000FF;">,</span><span style="color: #000000;">#2f25</span><span style="color: #0000FF;">})</span>

<span style="color: #008080;">end</span> <span style="color: #008080;">for</span>

<span style="color: #7060A8;">close</span><span style="color: #0000FF;">(</span><span style="color: #000000;">fn</span><span style="color: #0000FF;">)</span>

 

{{out}}

=={{header|PicoLisp}}==

Pico Lisp uses UTF-8 until told otherwise.

(in "wordlist"

(while (char)

(process @))

 

=={{header|Python}}==

{{works with|Python|2.7}}

def get_next_character(f):

# note: assumes valid utf-8

for c in get_next_character(f):

print(c)

 

{{works with|Python|3}}

Python 3 simplifies the handling of text files since you can specify an encoding.

"""Reads one character from the given textfile"""

c = f.read(1)

with open("input.txt", encoding="utf-8") as f:

for c in get_next_character(f):

 

=={{header|Racket}}==

Don't we all love self reference?

#lang racket

; This file contains utf-8 charachters: λ, α, γ ...

(for ([c (in-port read-char (open-input-file "read-file.rkt"))])

(display c))

Output:

#lang racket

; This file contains utf-8 charachters: λ, α, γ ...

(for ([c (in-port read-char (open-input-file "read-file.rkt"))])

(display c))

 

=={{header|Raku}}==

 

To read a single character at a time from the Standard Input terminal; $*IN in Raku:

 

Or, from a file:

 

my $in = open( $filename, :r ) orelse .die;

 

 

=={{header|REXX}}==

<br>The task's requirement stated that '''EOF''' was to be returned upon reaching the end-of-file, so this programming example was written as a subroutine (procedure).

<br>Note that displaying of characters that may modify screen behavior such as tab usage, backspaces, line feeds, carriage returns, "bells" and others are suppressed, but their hexadecimal equivalents are displayed.

parse arg iFID . /*iFID: is the fileID to be read. */

/* [↓] show the file's contents. */

exit /*stick a fork in it, we're all done. */

/*──────────────────────────────────────────────────────────────────────────────────────*/

'''input''' &nbsp; file: &nbsp; '''ABC'''

<br>and was created by the DOS command (under Windows/XP): &nbsp; &nbsp; '''echo 123 [¬ a prime]> ABC'''

 

===version 2===

* 29.12.2013 Walter Pachl

* read one utf8 character at a time

Return c

 

output:

<pre>y 79

 

=={{header|Ring}}==

fp = fopen("C:\Ring\ReadMe.txt","r")

r = fgetc(fp)

end

fclose(fp)

Output:

<pre>

{{works with|Ruby|1.9}}

 

f.each_char{|c| p c}

 

or

 

while c = f.getc

p c

end

 

=={{header|Run BASIC}}==

numChars = 1 ' specify number of characters to read

a$ = input$(#f,numChars) ' read number of characters specified

b$ = input$(#f,1) ' read one character

 

=={{header|Rust}}==

originally.

 

convert::TryFrom,

fmt::{Debug, Display, Formatter},

 

Ok(())

 

 

the file [http://seed7.sourceforge.net/libraries/utf8.htm#STD_UTF8_OUT STD_UTF8_OUT] is used.

 

include "utf8.s7i";

 

close(inFile);

end if;

 

{{out}}

 

=={{header|Sidef}}==

var fh = file.open_r # equivalent with: file.open('<:utf8')

fh.each_char { |char|

printf("got character #{char} [U+%04x]\n", char.ord)

{{out}}

<pre>

=={{header|Smalltalk}}==

{{works with|Smalltalk/X}}

utfStream := 'input' asFilename readStream asUTF8EncodedStream.

[utfStream atEnd] whileFalse:[

Transcript showCR:'got char ',utfStream next.

].

 

=={{header|Tcl}}==

To read a single character from a file, use:

This will read multiple bytes sufficient to obtain a Unicode character if a suitable encoding has been configured on the channel. For binary channels, this will always consume exactly one byte. However, the low-level channel buffering logic may consume more than one byte (which only really matters where the channel is being handed on to another process and the channel is over a file descriptor that doesn't support the <tt>lseek</tt> OS call); the extent of buffering can be controlled via:

When the channel is only being accessed from Tcl (or via Tcl's C API) it is not normally necessary to adjust this option.

 

=={{header|Wren}}==

 

File.open("input.txt") { |file|

offset = offset + 1

}

 

=={{header|zkl}}==

zkl doesn't know much about UTF-8 or Unicode but is able to test whether a string or number is valid UTF-8 or not. This code uses that to build a state machine to decode a byte stream into UTF-8 characters.

s+=chr;

try{ s.len(8); return(s) }

catch{ if(s.len()>6) throw(__exception) } // 6 bytes max for UTF-8

return(Void.Again,s); // call me again with s & another character

Used to modify a zkl iterator, it can consume any stream-able (files, strings, lists, etc) and provides support for foreach, map, look ahead, push back, etc.

obj.walker(3) // read characters

.tweak(readUTF8c)

utf8Walker(s).walk().println();

 

foreach c in (utf8Walker(Data(Void,s,"\n"))){ print(c) }

 

{{out}}

<pre>

</pre>

If you wish to push a UTF-8 stream through one or more functions, you can use the same state machine:

{{out}}<pre>-->€123</pre>

and returns a list of the eight UTF-8 characters (with newline).

Or, if file "foo.txt" contains the characters:

produces the same result.