Asp Forum - sbcl does not seem to work with non-ascii chars

Eduardo Costa

1/26/2016 4:12:00 PM

I am trying to write programs using sbcl, but it seems that the compiler does not work properly when I use non ascii chars. I wrote a very simple program that just write the French expression "Voilá" on the screen. Here is the program:

------- File: calltest.k -------------
#!/bin/bash

# LANG=pt_PT.UTF-8
LC_ALL=en_US.UTF-8
LANG=pt_PT.UTF-8 /home/enemm845/bin/sbcl.iii --load /home/enemm845/public_html/lsp/test.lsp

---------- File: test.lsp --------
;;#!/home/enemm845/bin/sbcl.iii --script

;;(setf sb-impl::*default-external-format* :utf-8)

(format t "Content-Type: text/html~%~%")

(format t "
<html>
<title>Voila</title>
<meta http-equiv=\"Content-Type\" content=\"text/html;charset=utf-8\"/>
<body>
Volá
</body>
</html>
" )

(exit)

In file calltest.k, I tried about every combination of external format. LC_CTYPE, LC_ALL, etc. In the Lisp side, I tried both with *default-external-format* and without it.

I noticed that a lot of people are experiencing similar problems with sbcl. Therefore, I hope somebody has found a solution. For languages like French, I can write a program that replaces accented words with html code. For languages like Greek or Chinese, this kind of solution is not available.

3 Answers

excelemailer

1/26/2016 4:16:00 PM

On Tuesday, January 26, 2016 at 8:11:35 AM UTC-8, Eduardo Costa wrote:
> I am trying to write programs using sbcl, but it seems that the compiler does not work properly when I use non ascii chars. I wrote a very simple program that just write the French expression "Voilá" on the screen. Here is the program:
>
> ------- File: calltest.k -------------
> #!/bin/bash
>
> # LANG=pt_PT.UTF-8
> LC_ALL=en_US.UTF-8
> LANG=pt_PT.UTF-8 /home/enemm845/bin/sbcl.iii --load> /home/enemm845/public_html/lsp/test.lsp
>
>
> ---------- File: test.lsp --------
> ;;#!/home/enemm845/bin/sbcl.iii --script
>
> ;;(setf sb-impl::*default-external-format* :utf-8)
>
> (format t "Content-Type: text/html~%~%")
>
> (format t "
> <html>
> <title>Voila</title>
> <meta http-equiv=\"Content-Type\" content=\"text/html;charset=utf-8\"/>
> <body>
> Volá
> </body>
> </html>
> " )
>
> (exit)
>
> In file calltest.k, I tried about every combination of external format. LC_CTYPE, LC_ALL, etc. In the Lisp side, I tried both with *default-external-format* and without it.
>
> I noticed that a lot of people are experiencing similar problems with sbcl. Therefore, I hope somebody has found a solution. For languages like French, I can write a program that replaces accented words with html code. For languages like Greek or Chinese, this kind of solution is not available.

I'm a lisp n00b, but when I had to convert an HTML file with non ascii characters, UTF-8 and UTF-8B didn't work, but iso-8859-1 did. So I would recommend trying iso-8859-1 until someone comes by with a better answer.

Pascal J. Bourguignon

1/26/2016 8:24:00 PM

Eduardo Costa <edu500ac@gmail.com> writes:

> I am trying to write programs using sbcl, but it seems that the
> compiler does not work properly when I use non ascii chars. I wrote a
> very simple program that just write the French expression "VoilÃ¡" on
> the screen.

Well, it's "VoilÃ ". There's no acute 'a' in French.

> Here is the program:
>
> ------- File: calltest.k -------------
> #!/bin/bash
>
> # LANG=pt_PT.UTF-8
> LC_ALL=en_US.UTF-8
> LANG=pt_PT.UTF-8 /home/enemm845/bin/sbcl.iii --load> /home/enemm845/public_html/lsp/test.lsp

I would add:

export LC_ALL
export LANG

> ---------- File: test.lsp --------
> ;;#!/home/enemm845/bin/sbcl.iii --script
>
> ;;(setf sb-impl::*default-external-format* :utf-8)
>
> (format t "Content-Type: text/html~%~%")
>
> (format t "
> <html>
> <title>Voila</title>
> <meta http-equiv=\"Content-Type\" content=\"text/html;charset=utf-8\"/>
> <body>
> VolÃ¡
> </body>
> </html>
> " )
>
> (exit)
>
> In file calltest.k, I tried about every combination of external
> format. LC_CTYPE, LC_ALL, etc. In the Lisp side, I tried both with
> *default-external-format* and without it.
>
> I noticed that a lot of people are experiencing similar problems with
> sbcl. Therefore, I hope somebody has found a solution. For languages
> like French, I can write a program that replaces accented words with
> html code. For languages like Greek or Chinese, this kind of solution
> is not available.

(setf SB-IMPL::*DEFAULT-EXTERNAL-FORMAT* :utf-8)

definitely should help, as:

export LANG=en_US.UTF_8

---(calltest.k)---------------------------------------------------------
#!/bin/bash
# -*- mode:shell-script;coding:utf-8 -*-
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
cat >/tmp/test.lisp <<'EOF'
;; -*- mode:lisp;coding:utf-8 -*-
(setf sb-impl::*default-external-format* :utf-8)
(with-open-file (*standard-output* "/tmp/test.html"
:direction :output
:if-does-not-exist :create
:if-exists :supersede)
(write-string "Content-Type: text/html

")
(write-string "
<html>
<title>VoilÃ </title>
<meta http-equiv=\"Content-Type\" content=\"text/html;charset=utf-8\"/>
<body>
VoilÃ , c'est l'Ã©tÃ© ! Ã?a se saurait, si Ã§a se fÃ®t.
</body>
</html>
"))
EOF
sbcl --no-userinit --noprint --noinform --load test.lisp < /dev/null > /dev/null
firefox --new-tab 'file:///tmp/test.html' >/dev/null 2>&1
------------------------------------------------------------------------
--
__Pascal Bourguignon__ http://www.informat...
â??The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.â? -- Carl Bass CEO Autodesk

Eduardo Costa

1/26/2016 8:29:00 PM

On Tuesday, January 26, 2016 at 2:15:58 PM UTC-2, excele...@gmail.com wrote:
> On Tuesday, January 26, 2016 at 8:11:35 AM UTC-8, Eduardo Costa wrote:
> > I am trying to write programs using sbcl, but it seems that the compiler does not work properly when I use non ascii chars. I wrote a very simple program that just write the French expression "Voilá" on the screen. Here is the program:
> >
> > ------- File: calltest.k -------------
> > #!/bin/bash
> >
> > # LANG=pt_PT.UTF-8
> > LC_ALL=en_US.UTF-8
> > LANG=pt_PT.UTF-8 /home/enemm845/bin/sbcl.iii --load> > /home/enemm845/public_html/lsp/test.lsp
> >
> >
> > ---------- File: test.lsp --------
> > ;;#!/home/enemm845/bin/sbcl.iii --script
> >
> > ;;(setf sb-impl::*default-external-format* :utf-8)
> >
> > (format t "Content-Type: text/html~%~%")
> >
> > (format t "
> > <html>
> > <title>Voila</title>
> > <meta http-equiv=\"Content-Type\" content=\"text/html;charset=utf-8\"/>
> > <body>
> > Volá
> > </body>
> > </html>
> > " )
> >
> > (exit)
> >
> > In file calltest.k, I tried about every combination of external format. LC_CTYPE, LC_ALL, etc. In the Lisp side, I tried both with *default-external-format* and without it.
> >
> > I noticed that a lot of people are experiencing similar problems with sbcl. Therefore, I hope somebody has found a solution. For languages like French, I can write a program that replaces accented words with html code. For languages like Greek or Chinese, this kind of solution is not available.
>
> I'm a lisp n00b, but when I had to convert an HTML file with non ascii characters, UTF-8 and UTF-8B didn't work, but iso-8859-1 did. So I would recommend trying iso-8859-1 until someone comes by with a better answer.

I tried iso as well, with similar results. Could you check what I did wrong?

;;#!/home/enemm845/bin/sbcl.iii --script

(setf sb-impl::*default-external-format* :iso-8859-1)

(format t "Content-Type: text/html~%~%")

(format t "
<html>
<title>Voila</title>
<meta http-equiv=\"Content-Type\" content=\"text/html;charset=iso-8859-1\"/>
<body>
Volá
</body>
</html>
" )

(exit)

#!/bin/bash

# LANG=pt_PT.UTF-8
LC_ALL=pt_PT.iso-8859-1
export LC_ALL
/home/enemm845/bin/sbcl.iii --load /home/enemm845/public_html/lsp/test.lsp

comp.lang.lisp

sbcl does not seem to work with non-ascii chars

Eduardo Costa

excelemailer

Pascal J. Bourguignon

Eduardo Costa

x Login to ForumsZone