Modify

Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#1239 closed request (support)

Hungarian character support

Reported by: mihaly.reg@… Owned by:
Priority: normal Milestone:
Component: fon-base-firmware Version: 2.3.7.0 beta3
Severity: unknown
Cc: Hardware: 2.0n (FON2300)

Description

Accessing fonera Disc-A1 by ssh or winscp some filenames are shown with invalid characters, when filename contains hungarian characters like őúéáű. It happens usually in torrents. When file copied to windows the error is preserved, ie. the filename is with same strange characters. I think it's not 'windows fault, because if I download that file from the internet directly to PC it looks ok. Attached some examples.

Attachments (6)

HUNGARIAN_ssh_192.168.0.2_20121228_233812.log (4.6 KB) - added by mihaly.reg@… 6 years ago.
HUNGARIAN_WINSCP_2012-12-29_000927.jpg (49.3 KB) - added by mihaly.reg@… 6 years ago.
webpage_file_browser_2012-12-29_002512.jpg (178.5 KB) - added by mihaly.reg@… 6 years ago.
samba_2012-12-29_002725.jpg (158.7 KB) - added by mihaly.reg@… 6 years ago.
test1.zip (158.6 KB) - added by mihaly.reg@… 6 years ago.
test4.zip (72.4 KB) - added by mihaly.reg@… 6 years ago.

Download all attachments as: .zip

Change History (26)

Changed 6 years ago by mihaly.reg@…

Changed 6 years ago by mihaly.reg@…

comment:1 Changed 6 years ago by mihaly.reg@…

Upgraded to 2.3.7.0_DEV Paco (28 december 2012), the same wrong characters are displayed in ssh and winscp. The files are the same as for previous test, I have not tested a new download of files with HU characters.

comment:2 Changed 6 years ago by mihaly.reg@…

Note: in web browser firefox, page File Browser, the hungarian file names are displayed correctly ! Also looks fine in Samba ! The problem is only for SSH (putty) and WinSCP. Attached some screenshots for webpage and samba.

Changed 6 years ago by mihaly.reg@…

Changed 6 years ago by mihaly.reg@…

comment:3 Changed 6 years ago by matthijs

  • Milestone set to Firmware 2.3
  • Status changed from new to infoneeded

Just for testing, could you try uploading a file with these "strange" characters using WinSCP and using Samba and see what happens?

comment:4 Changed 6 years ago by mihaly.reg@…

test1: created on the computer a file with name: őúéáűöüó_hungarian.rtf, uploaded it using SCP to .../disc-a1/tmp/. Result: hungarian characters are displayed correctly in SCP and SSH! test attached: test1-zip.

It looks like it's transmission's fault. The filenames downloaded with transmission are wrong. Could this be actually transmission's fault ?

(I could not test with samba. For some strange reason samba stopped working on my fonera since I upgraded to Paco. I can see fonera in windows explorer, but access is denied to subfolders of Disc-A1. User is fonero and pass the same as on ssh. This error happened in the past also...but I can't remember how I solved it...)

Changed 6 years ago by mihaly.reg@…

comment:5 Changed 6 years ago by matthijs

The files you uploaded using SCP, are the filenames shown correctly in the webui file browser as well?

As for your samba problem, perhaps it helps to change the password in the Fonera GUI, possibly the samba password store (which is separate) somehow got out of sync.

comment:6 Changed 6 years ago by mihaly.reg@…

test2: I have upgraded also transmission on fonera (in all previous tests it was the transmission that came with beta3) to 2.71(13538) and downloaded again a hungarian torrent from a tracker. The error is the same, the folder/file names on fonera are created with strange characters visible with ssh and winscp. But! in fonera webpage file browser it shows corretly the hu characters! Also in Transmission Web Interface is OK! The problem is only for listing the content of fonera disc on console (ssh) or winscp.

comment:7 Changed 6 years ago by mihaly.reg@…

test3: just to be sure... I have put the same torrent in windows xp utorrent and the file/folder names are created correctly when the torrent begins download.

comment:8 Changed 6 years ago by mihaly.reg@…

To your question: A file created on Win XP with HU characters, uploaded to fonera with WinSCP is NOT shown correctly in foneras own filebrowser. See test4.zip. Could you try also? I have attached the file.

Changed 6 years ago by mihaly.reg@…

comment:9 Changed 6 years ago by mihaly.reg@…

Notes: when uploading this fileneme "őúéáűöüó_hungarian.rtf" from winxp to fonera by winscp the characters ő and ű becomes o and u in fonera browser ! All other characters become invalid, like this: o���u���_hungarian.rtf. But in WinSCP the look like this: oúéáuöüó_hungarian.rtf (seee that o and u were changed, all others not). And finally in ssh looks like this: oúéáuöüó_hungarian.rtf. Same as in WinSCP.

comment:10 Changed 6 years ago by mihaly.reg@…

To your suggestion for samba: it does not work. I have changed the root password (from webui, system, passwd), rebooted, but still denied:

C:\>net use R:
192.168.0.2\Disc-A1 /user:fonero my ...passwd System error 5 has occurred. Access is denied.

This is a new error in Paco? Anybody reported it? Could you try? Thanks!

comment:11 Changed 6 years ago by mihaly.reg@…

Note: the samba error is reported also by user Marzio Spairani on blog.

comment:12 Changed 6 years ago by matthijs

Ok, I think the issue is definitely with SSH/SCP, looking at your observations. I'm wondering if this might be a character set issue, that the Fonera is providing filenames in UTF-8 while PuTTY / WinSCP is expected Latin-1 / ISO8859-1.

Could you see if you can configure PuTTY and/or WinSCP for UTF-8 operation (IIRC there is a "translation" setting for PuTTY somewhere that you can set to UTF-8).

Normally, when your PuTTY / WinSCP is configured incorrectly, I think the shell or SSH server should take care of translating the filenames into the right character set. But I suspect that either busybox (the shell) or dropbear (the SSH server) does not support this translation, leading to the wrong characters you are seeing.

As for the samba issue, could you create a separate ticket for this? Be sure to note from which version you are upgrading from and add the /etc/config/samba file as well.

comment:13 Changed 6 years ago by mihaly.reg@…

Good news for SSH, I have set UTF-8 in putty options/translation and now the filenames show up right! (it was ISO-8859-1 before, as default). Thanks for the hint !

However I did not found in WinSCP the right setting for UTF8. There is a setting in Environment UTF8 encoding which is Auto/On/Off? but it does not change anything on behavior.

I should try also the FTP protocol...

comment:14 Changed 6 years ago by mihaly.reg@…

Great, with FTP protocol (using WinSCP) it's ok! So the only issue is SCP protocol.

comment:15 Changed 6 years ago by matthijs

  • Status changed from infoneeded to investigate

Great!

For WinSCP, it seems there is an option available:

http://winscp.net/eng/docs/faq_utf8

comment:16 Changed 6 years ago by mihaly.reg@…

It says that "For SCP protocol filename encoding conversion is not supported at all". This explains why I have the problem only on SCP protocol.

comment:17 Changed 6 years ago by matthijs

It also suggests that UTF-8 is the default, so if WinSCP uses UTF-8 for display and the server transfers using UTF-8, no conversion is needed.

Also, I was going to suggest SFTP, but I just realized SFTP is not supported by the Fonera.

comment:18 Changed 6 years ago by tman <mihaly.reg@…>

I'm not convinced that UTF-8 is default.

comment:19 Changed 6 years ago by matthijs

  • Milestone Firmware 2.3 deleted
  • Resolution set to support
  • Status changed from investigate to closed
  • Type changed from bug to request

I looked a bit more closely, it seems that even on my normal Linux machine, no translation between character sets is done: If my filesystem is mounted to use utf8 filenames, my terminal should be configured to display utf-8. I think this means we cannot really expect special characters to work, when the SSH/SCP client is not configured for utf-8.

As for WinSCP, it apparently does not support UTF-8 in SCP mode yet, only for SFTP, see this ticket. I assume it will display as Latin-1 now, but I'm not entirely sure.

Also note that we really need utf-8 to be able to handle all special characters. Even with fancy character set translation, there is only a very limited set of special characters that can be expressed in latin-1, you really need utf-8 for the rest.

This means there's really nothing we can do on the Fonera side to make this work by default, or at all with WinSCP.

I'm closing this ticket. If you have further questions, feel free to leave a comment.

comment:20 Changed 6 years ago by tman <mihaly.reg@…>

It's ok, is scp protocol fault not fonera.

Add Comment

Modify Ticket

Action
as closed The ticket will remain with no owner.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.