The eccoMAGIC Forums - How to Get rid of "Hidden Line Breaks"

Admin

Administrator

I love Ecco!

Posts: 134

How to Get rid of "Hidden Line Breaks"
05/31/08 at 16:35:17

Add this rule to affected folders, run the rule & remove it.

(turn off other rules you don't want running when you do this!)

Code:

++:!+: replace(gfv("E-Mail"), "\x0D(?!\x0A)","",1):

In this example, the "E-Mail" folder is used. The rule should be placed in the E-Mail folder !!

a 'better' rule is:

Code:

++:!+: replace(gfv("E-Mail"), "\x0D(?!\x0A)","",1): [E-Mail] ~= "\x0D(?!\x0A)"

How this works ?

The 'hidden line break' is a hex 0D,

0x0D followed by 0x0A is a NOT HIDDEN LINE FEED

so,

we want to kill ONLY 0x0D by itself.

Slang rule allows touching HEX VALUES DIRECTLY. (which is pretty cool).

hope helps!

Back to top

The Administrator.

IP Logged

rb

Senior Member

I Love Ecco!

Posts: 88

Re: How to Get rid of "Hidden Line Breaks"
Reply #1 - 06/01/08 at 21:36:58

1. How can I see these hidden line breaks in Ecco?
Per Yahoo forum, I need do setup Windows for full unicode alt charset. I checked in
control Panel | Regional and Language Options, but it was not clear how to change my setup for this..?

2. I have these hidden line breaks in several folders. I need to run the rule across ALL folders. Maybe better to use a LUA script that works on ALL folders (I haven't tried it yet. Wanted to see the hidden line breaks first...)

Back to top

IP Logged

Admin

Administrator

I love Ecco!

Posts: 134

Re: How to Get rid of "Hidden Line Breaks"
Reply #2 - 06/02/08 at 09:02:25

Quote from rb on 06/01/08 at 21:36:58:

1. How can I see these hidden line breaks in Ecco?
Per Yahoo forum, I need do setup Windows for full unicode alt charset. I checked in
control Panel | Regional and Language Options, but it was not clear how to change my setup for this..?

for example:

install Chinese, and set Chinese to be default for non-unicode windows.

but much effort just to 'see' 0x0D character in your data !!

Quote:

2. I have these hidden line breaks in several folders. I need to run the rule across ALL folders. Maybe better to use a LUA script that works on ALL folders (I haven't tried it yet. Wanted to see the hidden line breaks first...)

how did they get there ?

(cut and paste rule several times seems not so hard to do...

but LUA script would be nice;

or perl... etc..

)

Back to top

The Administrator.

IP Logged

rb

Senior Member

I Love Ecco!

Posts: 88

Re: How to Get rid of "Hidden Line Breaks"
Reply #3 - 06/02/08 at 19:09:58

Quote from Admin on 06/02/08 at 09:02:25:

for example:

install Chinese, and set Chinese to be default for non-unicode windows.

but much effort just to 'see' 0x0D character in your data !!

how did they get there ?

I am not sure how the hidden line breaks got there. Possibly many years ago when importing data. It is necessary for me to be able to see these hidden line brakes as I would like to be able to delete them manually. However, that is very difficult to do when I can't even see them. The line breaks could be in any of my folders (and I have a lot of folders). If I see the line breaks in the phone book form it would easy to delete.

I believe that I have installed Chinese already as I can read Chinese and in the Language tab (WIN XP SP2), the "Install Languages for East Asian Languages" box is checked. However, I can find no option to "set Chinese to be default for non-unicode windows" in the "Regional Options" or Languages" Tabs.

Back to top

IP Logged

Admin Administrator I love Ecco! Posts: 134	Re: How to Get rid of "Hidden Line Breaks" Reply #4 - 06/03/08 at 05:28:40 In WinXP at least, Regional and Language Options > Advanced > Language for non-Unicode programs if Chinese not work (depends on how you've set up) try Greek or Hebrew. keep in mind, only will show you on the phonebook SIDEBAR based on the fields selected in the current Phonebook default form.
Back to top	The Administrator. IP Logged

Admin Administrator I love Ecco! Posts: 134	Re: How to Get rid of "Hidden Line Breaks" Reply #5 - 06/03/08 at 05:32:51 Quote: It is necessary for me to be able to see these hidden line brakes as I would like to be able to delete them manually. Indeed nice to be able to access, see, and control, but in this case, why not just use the Slang Rule ? Unless you've got non-text data (seems unlikely) should be safe & easy fix.
Back to top	The Administrator. IP Logged

rb

Senior Member

I Love Ecco!

Posts: 88

Re: How to Get rid of "Hidden Line Breaks"
Reply #6 - 06/04/08 at 03:28:04

Quote from Admin on 06/03/08 at 05:28:40:

In WinXP at least,

Regional and Language Options > Advanced > Language for non-Unicode programs

if Chinese not work (depends on how you've set up)

try Greek or Hebrew.

keep in mind, only will show you on the phonebook SIDEBAR based on the fields selected in the current Phonebook default form.

I tried Chinese and Greek. With Greek I got the language bar.

Unfortunately neither worked. I can still not see the hidden line breaks after the E-mail address (for example) in the Phonebook form. However, if I copy/paste the E-mail to Notepad, the line break is copied and shows as a true line break.

Back to top

IP Logged

Alec_Burgess

EccoMagic Apprentice

We all Love Ecco!

Posts: 0
Gender: male

Re: How to Get rid of "Hidden Line Breaks"
Reply #7 - 06/04/08 at 07:17:58

Quote:

I tried Chinese and Greek. With Greek I got the language bar.

Unfortunately neither worked. I can still not see the hidden line breaks after the E-mail address (for example) in the Phonebook form.

rb - So current problem is that these hidden linebreaks are few enough that if you could identify them you could clean them up manually?

If you export to CSV can they be recognized at all?

I'm not familiar enough with either eccoext or LUA (basically not at all) but I'm pretty good with regular expression and Notetab (clips).

If you can post a sample ECCO file containing at least a couple of "hidden line breaks" or can post a "recipe" for creating them myself (that may be at the beginning of this thread and I've missed it) I'll have a crack at finding a way to at least identify them all (and possibly fixing them with Dave G's tools though this would be a stretch - I and you!! wouldn't want to make the cure worse than the problem)

Back to top

Regards ... Alec

IP Logged

Admin

Administrator

I love Ecco!

Posts: 134

Re: How to Get rid of "Hidden Line Breaks"
Reply #8 - 06/04/08 at 13:00:33

Quote from rb on 06/04/08 at 03:28:04:

I tried Chinese and Greek. With Greek I got the language bar.

Unfortunately neither worked.

there are multiple Chinese setup options, Greek should work, but depending on setup, conceivable that would not.

Enable East Asian language and RTL language support. Try Hebrew. It's dos codepage control is RTL, so is very good about taking over the display!

@rb, can post copy of file here (to thread) if helpful for others to follow. (DO NOT HAVE FILE OPEN WHEN ATTEMPTING UPLOAD.. it won't work.)

@alec,

can use perl, python, JavaScript, or VBScript also as slang 'rule' code. (IF the 'active script' version of any of those is installed on your system).

Back to top

The Administrator.

IP Logged

rb

Senior Member

I Love Ecco!

Posts: 88

Re: How to Get rid of "Hidden Line Breaks"
Reply #9 - 06/04/08 at 19:07:04

Quote from Alec_Burgess on 06/04/08 at 07:17:58:

rb - So current problem is that these hidden linebreaks are few enough that if you could identify them you could clean them up manually?

If you export to CSV can they be recognized at all?

I'm not familiar enough with either eccoext or LUA (basically not at all) but I'm pretty good with regular expression and Notetab (clips).

If you can post a sample ECCO file containing at least a couple of "hidden line breaks" or can post a "recipe" for creating them myself (that may be at the beginning of this thread and I've missed it) I'll have a crack at finding a way to at least identify them all (and possibly fixing them with Dave G's tools though this would be a stretch - I and you!! wouldn't want to make the cure worse than the problem)

I have uploaded an example file (the same one as to the yahoo forum) which contains two phonebook entries. In this case the hidden line breaks are after the .com in the E-mail address, and this causes extra lines in the phone book.

In my real file, I think that I may have removed hidden line breaks, but I still have extra lines in the phonebook. However, I can not be sure that all line breaks are removed since I can not see them.... Alas, I would like to see where there may be any leftover hidden line breaks.

Back to top

BlankLines2.eco

IP Logged

Admin Administrator I love Ecco! Posts: 134	Re: How to Get rid of "Hidden Line Breaks" Reply #10 - 06/05/08 at 07:50:14 change the slang rule to replace line break with "LINE BREAK" and then, alas, you'll see is there were any!
Back to top	The Administrator. IP Logged

Admin Administrator I love Ecco! Posts: 134	Re: How to Get rid of "Hidden Line Breaks" Reply #11 - 06/05/08 at 07:51:19 Quote from rb on 06/04/08 at 19:07:04: I still have extra lines in the phonebook. However, I can not be sure that all line breaks are removed since I can not see them.... Alas, I would like to see where there may be any leftover hidden line breaks. what you mean "extra lines" in phonebook ?
Back to top	The Administrator. IP Logged

rb

Senior Member

I Love Ecco!

Posts: 88

Re: How to Get rid of "Hidden Line Breaks"
Reply #12 - 06/05/08 at 14:04:26

Quote from Admin on 06/05/08 at 07:51:19:

.

what you mean "extra lines" in phonebook ?

Under each name Entry (item) there are one or more blank lines (some of my entries have up to 4 blank lines). See the attached file above for a 1 blank line example.

I just got it to work after installing RTL and Hebrew. Can now see the hidden line breaks. However, I just realized that I also have regular line breaks after the text in some columns/folders. They do not show up with a bogus character of course. Thus, I am now manually going through each column text value to remove real and hidden line breaks undecided

Back to top

IP Logged

Admin Administrator I love Ecco! Posts: 134	Re: How to Get rid of "Hidden Line Breaks" Reply #13 - 06/06/08 at 07:05:54 if you want to remove ALL line breaks, you can modify the slang rule to do that. Code: ++:!+: replace(gfv("FOLDER NAME HERE!"), "(\x0D\|\x0A)+","",1): should do the job for you [untested].
Back to top	The Administrator. IP Logged