[lnkForumImage]
TotalShareware - Download Free Software

Confronta i prezzi di migliaia di prodotti.
Asp Forum
 Home | Login | Register | Search 


 

Forums >

comp.lang.ruby

Re: Gateway Upgraded

James Gray

10/31/2007 1:48:00 PM

On Oct 29, 2007, at 7:37 PM, James Edward Gray II wrote:

> We are now running on my new gateway code that will hopefully get
> even more messages across the Usenet divide.

The TMail team tried to squeeze in a fix for our new gateway script
and I tried another deploy this morning. Unfortunately, it still
didn't work for our needs. We ran into the same issue in a different
place.

I've reverted the gateway code again. I was watching for the issue
this time and caught it fast. I believe only one message was not gated.

I apologize again for the continuing issues. We'll get this stuff
sorted out eventually and hopefully the new features will have been
worth the effort.

James Edward Gray II


15 Answers

James Gray

11/1/2007 12:34:00 PM

0

The new gateway is now in place and does seem to be gating messages.
Hooray for the fixes added to TMail just for us!

I'll try to monitor the gateway a bit over the next couple of days to
make sure it's doing its job. Help me out though and email me if you
notice oddities.

Thanks all. Sorry this took so long.

James Edward Gray II

James Gray

11/2/2007 12:49:00 PM

0

On Nov 1, 2007, at 7:34 AM, James Edward Gray II wrote:

> The new gateway is now in place and does seem to be gating messages.

Just a small update on how this wentâ?¦

According to my scan of the log this morning, every email the gateway
sent since yesterday's upgrade was accepted by Usenet. I was even
able to find messages modified by the gateway and they looked great
on comp.lang.ruby.

This transition appears to be a success.

James Edward Gray II


Arlen Cuss

11/2/2007 1:28:00 PM

0

Hi,

On Fri, 2007-11-02 at 21:48 +0900, James Edward Gray II wrote:
> Just a small update on how this wentâ?¦

Thank you.

>
> According to my scan of the log this morning, every email the gateway
> sent since yesterday's upgrade was accepted by Usenet. I was even
> able to find messages modified by the gateway and they looked great
> on comp.lang.ruby.
>
> This transition appears to be a success.
>
> James Edward Gray II
>
>

Arlen


Mikel Lindsaar

11/3/2007 7:48:00 AM

0

Thanks for your work James.

Mikel

On 11/3/07, Arlen Christian Mart Cuss <celtic@sairyx.org> wrote:
> Hi,
>
> On Fri, 2007-11-02 at 21:48 +0900, James Edward Gray II wrote:
> > Just a small update on how this wentâ?¦
>
> Thank you.
>
> >
> > According to my scan of the log this morning, every email the gateway
> > sent since yesterday's upgrade was accepted by Usenet. I was even
> > able to find messages modified by the gateway and they looked great
> > on comp.lang.ruby.
> >
> > This transition appears to be a success.
> >
> > James Edward Gray II
> >
> >
>
> Arlen
>
>
>

see

11/3/2007 3:30:00 PM

0

In article <57a815bf0711030047x813fe18r6f6d79c1be251092@mail.gmail.com>,
raasdnil@gmail.com writes:
> Thanks for your work James.
>
> > > According to my scan of the log this morning, every email the gateway
> > > sent since yesterday's upgrade was accepted by Usenet. I was even
> > > able to find messages modified by the gateway and they looked great
> > > on comp.lang.ruby.
> > >
> > > This transition appears to be a success.
> > >
> > > James Edward Gray II

Well, there appears to be at least one little glitch remaining.
The following header line in the immediately referenced article
shouldn't appear with an unencoded article body:

Content-Transfer-Encoding: Base64

It's possible that the error was in the originally submitted
message, but I'd guess that it's more likely that the gateway
translated the body but failed to update the header. In any case,
here's some more of the header for message identification:

From: raasdnil@gmail.com
Newsgroups: comp.lang.ruby
Subject: Re: Gateway Upgraded
Date: Sat, 3 Nov 2007 02:47:50 -0500
Message-ID: <57a815bf0711030047x813fe18r6f6d79c1be251092@mail.gmail.com>
X-Trace: talisker.lacave.net 1194076112 58526 65.111.164.187 (3 Nov 2007 07:48:32 GMT)
NNTP-Posting-Date: Sat, 3 Nov 2007 07:48:32 +0000 (UTC)
X-Mail-Count: 277342
X-Ml-Name: ruby-talk
X-Rubymirror: Yes
X-Ruby-Talk: <57a815bf0711030047x813fe18r6f6d79c1be251092@mail.gmail.com>

And just as a side remark, I'll note that the following comment
should be wrapped at normal text width.

X-Received-From: This message has been automatically forwarded from the
ruby-talk mailing list by a gateway at comp.lang.ruby. If it is SPAM, it did
not originate at comp.lang.ruby. Please report the original sender, and not
us. Thanks! For more details about this gateway, please visit: http://blog.grayproductions.net/categories/t...

Thanks for maintaining the gateway,

- dmw

--
.. Douglas Wells . Connection Technologies .
.. Internet: -sp9804- -at - contek.com- .

James Gray

11/3/2007 9:23:00 PM

0

On Nov 3, 2007, at 11:10 AM, Douglas Wells wrote:

> In article
> <57a815bf0711030047x813fe18r6f6d79c1be251092@mail.gmail.com>,
> raasdnil@gmail.com writes:
>> Thanks for your work James.
>>
>>>> According to my scan of the log this morning, every email the
>>>> gateway
>>>> sent since yesterday's upgrade was accepted by Usenet. I was even
>>>> able to find messages modified by the gateway and they looked great
>>>> on comp.lang.ruby.
>>>>
>>>> This transition appears to be a success.
>>>>
>>>> James Edward Gray II
>
> Well, there appears to be at least one little glitch remaining.
> The following header line in the immediately referenced article
> shouldn't appear with an unencoded article body:
>
> Content-Transfer-Encoding: Base64

I'll look into this issue tomorrow. Thanks for pointing it out.

James Edward Gray II

James Gray

11/4/2007 4:50:00 PM

0

Mikel, I have some TMail questions for youâ?¦

On Nov 3, 2007, at 11:10 AM, Douglas Wells wrote:

> In article
> <57a815bf0711030047x813fe18r6f6d79c1be251092@mail.gmail.com>,
> raasdnil@gmail.com writes:
>> Thanks for your work James.
>>
>>>> According to my scan of the log this morning, every email the
>>>> gateway
>>>> sent since yesterday's upgrade was accepted by Usenet. I was even
>>>> able to find messages modified by the gateway and they looked great
>>>> on comp.lang.ruby.
>>>>
>>>> This transition appears to be a success.
>>>>
>>>> James Edward Gray II
>
> Well, there appears to be at least one little glitch remaining.
> The following header line in the immediately referenced article
> shouldn't appear with an unencoded article body:
>
> Content-Transfer-Encoding: Base64

This was caused by some code like:

incoming = TMail::Mail.parse($stdin.read)
outgoing = TMail::Mail.new

outgoing.transfer_encoding = incoming.transfer_encoding
outgoing.body = incoming.body

I assume that's because TMail is undoing the Base64 encoding for me
and not putting it back to match the transfer_encoding() of
outgoing. Do I have that right?

What's the best way for us to fix this? Should I not copy the
transfer encoding of the original message? Will TMail normalize all
of them for me? What does it normalize to? Maybe I should declare
that.

Note that the affected message here was not one the gateway does any
translation for, so this is going to be a widespread issue with the
new gateway and we need to find a sensible solution for it.

> And just as a side remark, I'll note that the following comment
> should be wrapped at normal text width.
>
> X-Received-From: This message has been automatically forwarded
> from the
> ruby-talk mailing list by a gateway at comp.lang.ruby. If
> it is SPAM, it did
> not originate at comp.lang.ruby. Please report the original
> sender, and not
> us. Thanks! For more details about this gateway, please
> visit: http://blog.grayproductions.net/categories/t...

Mikel, why is TMail not wrapping this? I know you added some checks
to ignore X-* headers, but shouldn't it only ignore them if they
aren't easily wrapped? This one has a lot of whitespace in it, so
it's wrappable, right?

Thanks for the help.

James Edward Gray II


Mikel Lindsaar

11/4/2007 11:46:00 PM

0

> > Content-Transfer-Encoding: Base64
>
> This was caused by some code like:
>
> incoming = TMail::Mail.parse($stdin.read)
> outgoing = TMail::Mail.new
>
> outgoing.transfer_encoding = incoming.transfer_encoding
> outgoing.body = incoming.body
>
> I assume that's because TMail is undoing the Base64 encoding for me
> and not putting it back to match the transfer_encoding() of
> outgoing. Do I have that right?

TMail does not handle the email body. In fact, it basically ignores
it beyond breaking it down by parts when you explicitly request it.

So this is what happens:

mail = TMail::Mail.load("my_base64_encoded_body_email")
body = mail.body # this will actually decode the body for you to be nice
mail.body = body # change the body text to an unencoded string
mail['Content-Transfer-Encoding'] == Base64 # => true

Basically, TMail has no way of knowning that the body text has changed
to a non Base64 string, it needs to be told that.

I *think*, in essence, this is what is happening on the gateway,
please correct me if I am wrong.

An (easier?) way, that preserves the body text and the transfer
encoding type would be to do:

outgoing_mail = TMail.parse($stdin.read)
outgoing_mail['x-header'] = "whatever you need to add"

Because when you do the outgoing.body = incomming.body TMail is being
nice and decoding it for you.

If you want the undecoded text and still do the assignment, you can do:

outgoingmail.body = incommingmail.quoted_body

I think I might make an alias for "undecoded_body" to "quoted_body" but anyway.

Is there any reason you are making two TMail objects instead of just
making one new one and parsing in the incomming string and then adding
what you need?


> What's the best way for us to fix this? Should I not copy the
> transfer encoding of the original message? Will TMail normalize all
> of them for me? What does it normalize to? Maybe I should declare
> that.

Just make one object and pull in the data and leave the body alone, I
think, would be the easiest.

> Mikel, why is TMail not wrapping this? I know you added some checks
> to ignore X-* headers, but shouldn't it only ignore them if they
> aren't easily wrapped? This one has a lot of whitespace in it, so
> it's wrappable, right?

Hmm... I originally put in stuff to ignore the X-Headers completely,
this smelt a bit bad so then I changed this to wrapping anything that
has over 78 characters total that has a whitespace which is more RFC
compliant.

So, now it doesn't ignore any headers... it tries to wrap them if they
have whitespace that it can wrap (ie, whitespace in the first 78
characters). If it doesn't have whitespace within the first 78 chars
(including the header) it doesn't wrap....

So, basically, this should be wrapping correctly.

But I am pretty sure you are using this version of trunk because that
is the same revision that fixed the reply-to problem you were having.

Could you send me an unencoded, unwrapped copy of the string you
insert so I can make a test case with it?

Regards

Mikel

James Gray

11/5/2007 3:37:00 AM

0

On Nov 4, 2007, at 5:46 PM, Mikel Lindsaar wrote:

> Is there any reason you are making two TMail objects instead of just
> making one new one and parsing in the incomming string and then adding
> what you need?

I use the two message approach because I filter what goes through.
Only certain headers are passed and we may need to translate the
content-type and body. It seems easier to work with a blank slate
than to try and remove all of the unwanted material.

James Edward Gray II

James Gray

11/5/2007 1:50:00 PM

0

On Nov 3, 2007, at 11:10 AM, Douglas Wells wrote:

> In article
> <57a815bf0711030047x813fe18r6f6d79c1be251092@mail.gmail.com>,
> raasdnil@gmail.com writes:
>> Thanks for your work James.
>>
>>>> According to my scan of the log this morning, every email the
>>>> gateway
>>>> sent since yesterday's upgrade was accepted by Usenet. I was even
>>>> able to find messages modified by the gateway and they looked great
>>>> on comp.lang.ruby.
>>>>
>>>> This transition appears to be a success.
>>>>
>>>> James Edward Gray II
>
> Well, there appears to be at least one little glitch remaining.
> The following header line in the immediately referenced article
> shouldn't appear with an unencoded article body:
>
> Content-Transfer-Encoding: Base64

I've deployed some fixes to the gateway this morning that I hope will
correct this issue. Please let me know if you see more problems.

James Edward Gray II