Using the ENTER instuction seems to create
problems when the amount %esp must be decreased
makes it change page !

  Does the page mecanism not know how to handle ENTER instruction ?
Is this a known i386 Linux problem ?
Is it solved in recent version ?

 Or is ENTER simply consider as an invalid instruction in Linux ?
This was tested on v2.0.33

Followup to:  <7hutm2$1f...@news.u-strasbg.fr>
By author:    mul...@ics.u-strasbg.fr (Pierre Muller)
In newsgroup: comp.os.linux.development.system

Define "problems".  Are you perhaps merely overflowing your allowed
stack space?

        -hpa
--
"The user's computer downloads the ActiveX code and simulates a 'Blue
Screen' crash, a generally benign event most users are familiar with
and that would not necessarily arouse suspicions."
-- Security exploit description on http://www.zks.net/p3/how.asp

  No,

   a simple assembler code like

    enter $0x3148,$0

at top most level is enough to get a SIGSEGV !
  Remark : I don't know how to create an executable from a single
assembler file (so my code is imbedded in FPC code !)

  I just want to stress that the value does not have to be over 0x1000
it's just if you pass a page boundary !

|>
|>   Using the ENTER instuction seems to create
|> problems when the amount %esp must be decreased
|> makes it change page !
|>
|>   Does the page mecanism not know how to handle ENTER instruction ?
|> Is this a known i386 Linux problem ?
|> Is it solved in recent version ?
|>
|>  Or is ENTER simply consider as an invalid instruction in Linux ?
|> This was tested on v2.0.33

One warning (found out on an P2): At least ENTER 0 (ie. just pushing ebp and
and setting it to esp) is more than 500-1000% SLOWER * than
'puhsl %ebp; movl %esp,%ebp'.  It seems that this instruction has a bug and/or
is executed in microcode. On the other hand, LEAVE is slightly faster than the
'dicrete' commands.

*) recursive Fibonacci(40) is twice the time with ENTER as without it, LEAVE
gains about 3%.
--
        Bye
         Georg Acher, ac...@in.tum.de        
         http://www.in.tum.de/~acher/
          "Oh no, not again !" The bowl of petunias          

On 19 May 1999, Pierre Muller wrote:

Kernel 2.2.7 still has the same problem.

I did some more tests and found out that in my case (2.2.7 on i486) the
problem only occurs when reserving more than 28 bytes of local storage
(ESP gets decremented by more than 32).

I think this may be related to the following code from the kernel
linux/arch/i386/fault.c (line 124) :
                /*
                 * accessing the stack below %esp is always a bug.
                 * The "+ 32" is there due to some instructions (like
                 * pusha) doing post-decrement on the stack and that
                 * doesn't show up until later..
                 */
                if (address + 32 < regs->esp)
                        goto bad_area;

The enter seems to check page availability for the entire stack region it
claims (which is odd since it only needs to access the upper 4 bytes,
but then again the intel docs do say that it checks the SS limit for the
entire region). After changing + 32 into + 64, I was able to reserve up to
60 bytes without crashing.

My guess is that Linux simply doesn't support the ENTER insn (I think GCC
doesn't use it).

Bye,
Joris van Rantwijk
J.F.vanRantw...@twi.tudelft.nl - http://deadlock.et.tudelft.nl/~joris/

  OK, the result of my discussion with H. Peter Anvin
is that ENTER is not handled correctly in the kernel
and thus a program that has an enter instruction that causes
esp to go to a page below creates a SIGSEGV !

  Thus ENTER should NOT be used under linux !!!

 My opinion was that such a bug should be fixed in the kernel,
but H. Peter Anvin disagreed with me on that point !
His argument was that ENTER is a useless instruction anyway !

   Sample code is :

   .globl _start
  _start
         enter $0x33cc,$0
         leave
         movl $1,%eax
         xorl %ebx,%ebx
         int $0x80

  as -o test.c test.s
  ld test.o
  ./a.out

  gdb ./a.out
  gdb>  run
   confirms that the SIGSEGV occurs on the ENTER instruction !

A year or two ago I had this problem with some ancient (I think
they were Coherent) binaries under iBCS which used enter...leave.
If you look at the Linux page fault handler you will find that
it treats any fault for an address more than a certain amount
below %esp to be an illegal dereference rather than a stack
expansion. The problem with enter is that it appears to write
save values _before_ setting the new value of %esp so enter
with a size greater then Linux's magic constant which cause
a page fault will generate a seg fault.

  I think one, maybe two people ever filed an iBCS bug report
about it so I never bothered to worry about a fix other than
hacking out the check from the page fault handler. Anyway, no
one has used enter...leave for years because doing it by hand
with simple instructions is usually a lot faster (and lets you
do the extra idiot check in the fault handler).

                                Mike

--
    A train stops at a train station, a bus stops at a bus station.
    On my desk I have a work station...
.----------------------------------------------------------------------.
|  Mike Jagdis                  |  Internet:  mailto:m...@roan.co.uk   |
|  Roan Technology Ltd.         |                                      |
|  54A Peach Street, Wokingham  |  Telephone:  +44 118 989 0403        |
|  RG40 1XG, ENGLAND            |  Fax:        +44 118 989 1195        |
`----------------------------------------------------------------------'

Followup to:  <7i3a1s$si...@news.u-strasbg.fr>
By author:    mul...@ics.u-strasbg.fr (Pierre Muller)
In newsgroup: comp.os.linux.development.system

Well, yes, it is: it is slower than the simple alternative which
doesn't cause the problem.  Hence, I conclude that "fixing" the
"problem" would just be bloat.  "Doctor, it hurts when I do this..."

        -hpa
--
"The user's computer downloads the ActiveX code and simulates a 'Blue
Screen' crash, a generally benign event most users are familiar with
and that would not necessarily arouse suspicions."
-- Security exploit description on http://www.zks.net/p3/how.asp

In article <Pine.LNX.3.96.990521100207.12723B-100...@deadlock.et.tudelft.nl>,
Joris van Rantwijk  <jo...@deadlock.et.tudelft.nl> wrote:

As has been mentioned by others: it's not a problem, it's a feature.

Linux will refuse to extend the stack beyond the stack pointer. Using
memory under the stack pointer is not considered acceptable, so you
should never do something like this (pseudocode):

        *(esp-128) = val;
        esp -= 128;

because it just sets you up for bugs when a signal comes in and trashes
what you had on the stack.

"enter" is a special, and very ugly x86 instruction that does exactly
the above. It so happens that we _could_ allow it, but as others have
mentioned, using "enter" in the first place is just stupid anyway, so
there really isn't any reason to allow that kind of braindamage.

Indeed.  Linux allows a small amount of slop underneath the stack
pointer, because "pusha" has the same problem, and for "pusha" there is
no good alternative way of doing the same thing.

Exactly. The comment pretty much says it all.

No. I think "enter" claims page availability for just the last byte it
claims, not the whole region.

"enter" is just basically a horrible crock, and should not be used in
any case. It's slower than the (much more natural) alternatives, and
generally just doesn't have any redeeming features at all.

Sure, you can use it, but you can't depend on Linux being nice to you
and extending the stack automatically.  If you extend the stack by hand
before using enter, you can then use it if you want (another way of
saying "if you really know what you're doing, Linux lets you shoot
yourself in the head if you want to"). The example could be something
like this:

        /* allocate 32kB of stack space */
        subl $32768,%esp
        movl $0,0(%esp)
        addl $32768,%esp

        /* now you can use enter to your hearts content */
        enter 256,0

but it's not as if I really see the reason for doing something like the
above ;)

                Linus