OutOfMemoryError PermGen Space...

I'm running Wicket 1.4.9 and Sun's JDK 1.6.0_20 on a RHEL4 64-bit machine.

On Thu, Jul 22, 2010 at 2:30 PM, James Carman

Post by James Carman
Guys, our production server is running into an error from time to
time. I get the below stack trace after the site has been running for
a while. It's not running in development mode. It's running in
deployment mode. Any thoughts? What's with this
generateSerializationConstructor() stuff?
sun.misc.Unsafe.defineClass(Native Method)
sun.reflect.ClassDefiner.defineClass(ClassDefiner.java:45)
sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:381)
java.security.AccessController.doPrivileged(Native Method)
sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:377)
sun.reflect.MethodAccessorGenerator.generateSerializationConstructor(MethodAccessorGenerator.java:95)
sun.reflect.ReflectionFactory.newConstructorForSerialization(ReflectionFactory.java:313)
java.io.ObjectStreamClass.getSerializableConstructor(ObjectStreamClass.java:1327)
java.io.ObjectStreamClass.access$1500(ObjectStreamClass.java:52)
java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:437)
java.security.AccessController.doPrivileged(Native Method)
java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:413)
java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:310)
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1106)
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
java.io.ObjectOutputStream.defaultWriteObject(ObjectOutputStream.java:416)
org.apache.wicket.Component.writeObject(Component.java:4438)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945)
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1461)
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1338)
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1146)
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

Martin Makundi

2010-07-22 19:25:50 UTC

Where is the outofmemoryerror ?

**
Martin

Post by James Carman
I'm running Wicket 1.4.9 and Sun's JDK 1.6.0_20 on a RHEL4 64-bit machine.
On Thu, Jul 22, 2010 at 2:30 PM, James Carman

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

James Carman

2010-07-22 19:28:11 UTC

Oops! I must not have copied that line. All it said was "PermGen space"

On Thu, Jul 22, 2010 at 3:25 PM, Martin Makundi

Post by Martin Makundi
Where is the outofmemoryerror ?
**
Martin

Post by James Carman
I'm running Wicket 1.4.9 and Sun's JDK 1.6.0_20 on a RHEL4 64-bit machine.
On Thu, Jul 22, 2010 at 2:30 PM, James Carman

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

Martin Makundi

2010-07-22 19:31:49 UTC

You did try to change your permgen setting?

http://rimuhosting.com/knowledgebase/linux/java/-Xmx-settings

**
Martin

Oops! I must not have copied that line. All it said was "PermGen space"
On Thu, Jul 22, 2010 at 3:25 PM, Martin Makundi

Post by Martin Makundi
Where is the outofmemoryerror ?
**
Martin

Post by James Carman
I'm running Wicket 1.4.9 and Sun's JDK 1.6.0_20 on a RHEL4 64-bit machine.
On Thu, Jul 22, 2010 at 2:30 PM, James Carman

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

James Carman

2010-07-22 19:33:20 UTC

Of course. It's set at 1024m! I bumped it from 256 to 512 to 1024
now. I'm still seeing the error.

On Thu, Jul 22, 2010 at 3:31 PM, Martin Makundi

Post by Martin Makundi
You did try to change your permgen setting?
http://rimuhosting.com/knowledgebase/linux/java/-Xmx-settings
**
Martin

Oops! I must not have copied that line. All it said was "PermGen space"
On Thu, Jul 22, 2010 at 3:25 PM, Martin Makundi

Post by Martin Makundi
Where is the outofmemoryerror ?
**
Martin

Post by James Carman
I'm running Wicket 1.4.9 and Sun's JDK 1.6.0_20 on a RHEL4 64-bit machine.
On Thu, Jul 22, 2010 at 2:30 PM, James Carman

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

david_

2010-07-22 19:47:09 UTC

http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html

<http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html>these
are my settings
set JAVA_OPTS=%JAVA_OPTS% -Xms512m -Xmx1024m -XX:PermSize=256m
-XX:MaxPermSize=256m

2010/7/22 James Carman [via Apache Wicket] <

Post by James Carman
Of course. It's set at 1024m! I bumped it from 256 to 512 to 1024
now. I'm still seeing the error.
On Thu, Jul 22, 2010 at 3:31 PM, Martin Makundi
<[hidden email] <http://user/SendEmail.jtp?type=node&node=2299232&i=0>>

Post by Martin Makundi
You did try to change your permgen setting?
http://rimuhosting.com/knowledgebase/linux/java/-Xmx-settings
**
Martin

Post by James Carman
Oops! I must not have copied that line. All it said was "PermGen

space"

Post by James Carman
On Thu, Jul 22, 2010 at 3:25 PM, Martin Makundi
<[hidden email] <http://user/SendEmail.jtp?type=node&node=2299232&i=2>>

Post by Martin Makundi
Where is the outofmemoryerror ?
**
Martin

Post by James Carman
I'm running Wicket 1.4.9 and Sun's JDK 1.6.0_20 on a RHEL4 64-bit

machine.

Post by James Carman
On Thu, Jul 22, 2010 at 2:30 PM, James Carman
<[hidden email]<http://user/SendEmail.jtp?type=node&node=2299232&i=4>>

Post by James Carman
Guys, our production server is running into an error from time to
time. I get the below stack trace after the site has been running

for

Post by James Carman
a while. It's not running in development mode. It's running in
deployment mode. Any thoughts? What's with this
generateSerializationConstructor() stuff?
sun.misc.Unsafe.defineClass(Native Method)
sun.reflect.ClassDefiner.defineClass(ClassDefiner.java:45)

sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:381)

Post by James Carman
java.security.AccessController.doPrivileged(Native Method)

sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:377)
sun.reflect.MethodAccessorGenerator.generateSerializationConstructor(MethodAccessorGenerator.java:95)
sun.reflect.ReflectionFactory.newConstructorForSerialization(ReflectionFactory.java:313)
java.io.ObjectStreamClass.getSerializableConstructor(ObjectStreamClass.java:1327)
java.io.ObjectStreamClass.access$1500(ObjectStreamClass.java:52)

Post by James Carman
java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:437)
java.security.AccessController.doPrivileged(Native Method)
java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:413)
java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:310)

java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1106)
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
java.io.ObjectOutputStream.defaultWriteObject(ObjectOutputStream.java:416)

Post by James Carman
org.apache.wicket.Component.writeObject(Component.java:4438)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

Post by James Carman
java.lang.reflect.Method.invoke(Method.java:597)

java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:945)
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1461)
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1338)
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1146)
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1509)
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1474)
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1392)
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1150)
java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:326)

Post by James Carman
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<http://user/SendEmail.jtp?type=node&node=2299232&i=5>
For additional commands, e-mail: [hidden email]<http://user/SendEmail.jtp?type=node&node=2299232&i=6>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<http://user/SendEmail.jtp?type=node&node=2299232&i=13>
For additional commands, e-mail: [hidden email]<http://user/SendEmail.jtp?type=node&node=2299232&i=14>
------------------------------
http://apache-wicket.1842946.n4.nabble.com/OutOfMemoryError-PermGen-Space-tp2299132p2299232.html
To start a new topic under Wicket - User, email
To unsubscribe from Wicket - User, click here<http://apache-wicket.1842946.n4.nabble.com/subscriptions/Unsubscribe.jtp?code=bWV1bGVtYW5zLmRhdmlkQGdtYWlsLmNvbXwxODQyOTQ3fC0xOTcyOTcxMjQ2>.

--
View this message in context: http://apache-wicket.1842946.n4.nabble.com/OutOfMemoryError-PermGen-Space-tp2299132p2299250.html
Sent from the Wicket - User mailing list archive at Nabble.com.

James Carman

2010-07-22 19:53:52 UTC

I'm running Tomcat, so it's:

$ env | grep CATALINA
CATALINA_OPTS= -Xmx4096m -Xms2048m -XX:MaxPermSize=1024m
-XX:+UseParallelGC -server

Post by david_
http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html
<http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html>these
are my settings
set JAVA_OPTS=%JAVA_OPTS% -Xms512m -Xmx1024m -XX:PermSize=256m
-XX:MaxPermSize=256m
2010/7/22 James Carman [via Apache Wicket] <

Of course. It's set at 1024m! I bumped it from 256 to 512 to 1024
now. I'm still seeing the error.
On Thu, Jul 22, 2010 at 3:31 PM, Martin Makundi
<[hidden email] <http://user/SendEmail.jtp?type=node&node=2299232&i=0>>

Post by Martin Makundi
You did try to change your permgen setting?
http://rimuhosting.com/knowledgebase/linux/java/-Xmx-settings
**
Martin

Oops! I must not have copied that line. All it said was "PermGen

space"

On Thu, Jul 22, 2010 at 3:25 PM, Martin Makundi
<[hidden email] <http://user/SendEmail.jtp?type=node&node=2299232&i=2>>

Post by Martin Makundi
Where is the outofmemoryerror ?
**
Martin

Post by James Carman
I'm running Wicket 1.4.9 and Sun's JDK 1.6.0_20 on a RHEL4 64-bit

machine.

Post by James Carman
On Thu, Jul 22, 2010 at 2:30 PM, James Carman
<[hidden email]<http://user/SendEmail.jtp?type=node&node=2299232&i=4>>

Post by James Carman
Guys, our production server is running into an error from time to
time. I get the below stack trace after the site has been running

for

sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:381)

Post by James Carman
java.security.AccessController.doPrivileged(Native Method)

Post by James Carman
org.apache.wicket.Component.writeObject(Component.java:4438)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

Post by James Carman
java.lang.reflect.Method.invoke(Method.java:597)

--
View this message in context: http://apache-wicket.1842946.n4.nabble.com/OutOfMemoryError-PermGen-Space-tp2299132p2299250.html
Sent from the Wicket - User mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

Andreas Prieß

2010-07-23 07:55:19 UTC

Post by James Carman
$ env | grep CATALINA
CATALINA_OPTS= -Xmx4096m -Xms2048m -XX:MaxPermSize=1024m
-XX:+UseParallelGC -server

Well, if you have increased memory for the permanent generation several
times and keep seeing this error, for me this sounds like a memory leak.

Tuning the java garbage collection will in all cases not help here,
because java only throws an out of memory error if garbage collection
does not help any more.

I would enable the following for tomcat:
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/log/java/whatever

Then you can analyse what holds all the memory after the next crash.

A nice tool for that is the Eclipse Memory Analyzer:
http://www.eclipse.org/mat/

A little how to for getting started:
http://wiki.eclipse.org/index.php/MemoryAnalyzer

And you should search what they have at the tomcat pages and on their
mailing list archives:
http://wiki.apache.org/tomcat/OutOfMemory

This is most likely not specific to wicket itself.

Andreas

Post by Martin Makundi
You did try to change your permgen setting?
http://rimuhosting.com/knowledgebase/linux/java/-Xmx-settings
**
Martin

Post by James Carman
Oops! I must not have copied that line. All it said was "PermGen

space"

Post by James Carman
On Thu, Jul 22, 2010 at 3:25 PM, Martin Makundi
<[hidden email] <http://user/SendEmail.jtp?type=node&node=2299232&i=2>>

Post by Martin Makundi
Where is the outofmemoryerror ?
**
Martin

Post by James Carman
I'm running Wicket 1.4.9 and Sun's JDK 1.6.0_20 on a RHEL4 64-bit

machine.

Post by James Carman
On Thu, Jul 22, 2010 at 2:30 PM, James Carman
<[hidden email]<http://user/SendEmail.jtp?type=node&node=2299232&i=4>>

Post by James Carman
Guys, our production server is running into an error from time to
time. I get the below stack trace after the site has been running

for

Post by James Carman
a while. It's not running in development mode. It's running in
deployment mode. Any thoughts? What's with this
generateSerializationConstructor() stuff?

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

Johan Compagner

2010-07-23 08:19:50 UTC

i think the problem with those memory analyzers that they give you the heap
Not the non heap, which is currently the problem..

If really constantly classes are leaked without that you constantly
redeploy web applications (because that would be a leak somewhere in
wicket or the app itself)
but the only thing you do is serializing these kind of objects:

// not serializeable and doesnt have a default constructor
public class A {
String name;

// add this constructor so the compiler doesnt generate a default one.
public A(String name) {
this.name = name;
}
}

public class B extends A implements Serializable {
int aNumber;
public B() {
super("B");
aNumber = 10;
}
}

If you then serialize instances of B then i think you will come into
the section of the stacktrace James gave.

But this shouldnt really leak into the none heap. (and result in an
never ending grow) that would just a bug in java itself if you ask me.
i think the work around is to give the A class a private constructor:

private A() {}

Then i think nothing is tried to be generated.

Post by Andreas PrieÃ

Post by James Carman
$ env | grep CATALINA
CATALINA_OPTS= -Xmx4096m -Xms2048m -XX:MaxPermSize=1024m
-XX:+UseParallelGC -server

Well, if you have increased memory for the permanent generation several
times and keep seeing this error, for me this sounds like a memory leak.
Tuning the java garbage collection will in all cases not help here,
because java only throws an out of memory error if garbage collection
does not help any more.
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/log/java/whatever
Then you can analyse what holds all the memory after the next crash.
http://www.eclipse.org/mat/
http://wiki.eclipse.org/index.php/MemoryAnalyzer
And you should search what they have at the tomcat pages and on their
http://wiki.apache.org/tomcat/OutOfMemory
This is most likely not specific to wicket itself.
Andreas

Post by Martin Makundi
You did try to change your permgen setting?
http://rimuhosting.com/knowledgebase/linux/java/-Xmx-settings
**
Martin

Oops! I must not have copied that line. All it said was "PermGen

space"

On Thu, Jul 22, 2010 at 3:25 PM, Martin Makundi
<[hidden email] <http://user/SendEmail.jtp?type=node&node=2299232&i=2>>

Post by Martin Makundi
Where is the outofmemoryerror ?
**
Martin

Post by James Carman
I'm running Wicket 1.4.9 and Sun's JDK 1.6.0_20 on a RHEL4 64-bit

machine.

Post by James Carman
On Thu, Jul 22, 2010 at 2:30 PM, James Carman
<[hidden email]<http://user/SendEmail.jtp?type=node&node=2299232&i=4>>

Post by James Carman
Guys, our production server is running into an error from time to
time. I get the below stack trace after the site has been running

for

Post by James Carman
a while. It's not running in development mode. It's running in
deployment mode. Any thoughts? What's with this
generateSerializationConstructor() stuff?

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

James Carman

2010-07-23 11:11:08 UTC

Johan,

This shouldn't happen if the number of class similar to "B" are
finite, correct (at least not with 1GB of permgen)? Java shouldn't be
generating constructors multiple times. There would be one
ObjectStreamClass for each type of object (as I understand the code),
since they use that lookup() method to find them. Or, could I somehow
be leaking classes (hibernate-generated subclasses of my entities,
proxy classes generated by wicket-spring, etc.)? That's going to be
harder to track down.

Post by Johan Compagner
i think the problem with those memory analyzers that they give you the heap
Not the non heap, which is currently the problem..
If really constantly classes are leaked without that you constantly
redeploy web applications (because that would be a leak somewhere in
wicket or the app itself)
// not serializeable and doesnt have a default constructor
public class A {
String name;
// add this constructor so the compiler doesnt generate a default one.
public A(String name) {
this.name = name;
}
}
public class B extends A implements Serializable {
int aNumber;
public B() {
super("B");
aNumber = 10;
}
}
If you then serialize instances of B then i think you will come into
the section of the stacktrace James gave.
But this shouldnt really leak into the none heap. (and result in an
never ending grow) that would just a bug in java itself if you ask me.
private A() {}
Then i think nothing is tried to be generated.

Post by Andreas PrieÃ

Post by James Carman
$ env | grep CATALINA
CATALINA_OPTS= -Xmx4096m -Xms2048m -XX:MaxPermSize=1024m
-XX:+UseParallelGC -server

Well, if you have increased memory for the permanent generation several
times and keep seeing this error, for me this sounds like a memory leak.
Tuning the java garbage collection will in all cases not help here,
because java only throws an out of memory error if garbage collection
does not help any more.
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/log/java/whatever
Then you can analyse what holds all the memory after the next crash.
http://www.eclipse.org/mat/
http://wiki.eclipse.org/index.php/MemoryAnalyzer
And you should search what they have at the tomcat pages and on their
http://wiki.apache.org/tomcat/OutOfMemory
This is most likely not specific to wicket itself.
Andreas

Post by Martin Makundi
You did try to change your permgen setting?
http://rimuhosting.com/knowledgebase/linux/java/-Xmx-settings
**
Martin

Oops! I must not have copied that line. All it said was "PermGen

space"

On Thu, Jul 22, 2010 at 3:25 PM, Martin Makundi
<[hidden email] <http://user/SendEmail.jtp?type=node&node=2299232&i=2>>

Post by Martin Makundi
Where is the outofmemoryerror ?
**
Martin

Post by James Carman
I'm running Wicket 1.4.9 and Sun's JDK 1.6.0_20 on a RHEL4 64-bit

machine.

Post by James Carman
On Thu, Jul 22, 2010 at 2:30 PM, James Carman
<[hidden email]<http://user/SendEmail.jtp?type=node&node=2299232&i=4>>

Post by James Carman
Guys, our production server is running into an error from time to
time. I get the below stack trace after the site has been running

for

Post by James Carman
a while. It's not running in development mode. It's running in
deployment mode. Any thoughts? What's with this
generateSerializationConstructor() stuff?

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

Andreas Prieß

2010-07-23 11:15:50 UTC

Post by Johan Compagner
i think the problem with those memory analyzers that they give you the heap
Not the non heap, which is currently the problem..

Yes, but they are the only thing you have if you do not use a profiler
to look at the running app.

And since the permanent generation does not grow on it's own without any
link to the heap, you can find the reason for the problem this way with
a little luck.

Post by Johan Compagner
If really constantly classes are leaked without that you constantly
redeploy web applications (because that would be a leak somewhere in
wicket or the app itself)

It can be a problem in any library that you use and that are generally
quite many these days...
I had similar problems with database drivers and object mapping libs,
but you are right, in this case the problem normally arises with
redeployments of the app without restart of the container.

http://dev.eclipse.org/blogs/memoryanalyzer/2008/05/17/the-unknown-generation-perm/
You may be right with your analysis of the serialization, I just want to
note here that in case of an out of memory error the stack trace can be
completely irrelevant. It just shows at what step the memory was full,
but that step does not have to be the cause for the memory filling up.

Post by Johan Compagner
// not serializeable and doesnt have a default constructor
public class A {
String name;
// add this constructor so the compiler doesnt generate a default one.
public A(String name) {
this.name = name;
}
}
public class B extends A implements Serializable {
int aNumber;
public B() {
super("B");
aNumber = 10;
}
}
If you then serialize instances of B then i think you will come into
the section of the stacktrace James gave.
But this shouldnt really leak into the none heap. (and result in an
never ending grow) that would just a bug in java itself if you ask me.
private A() {}
Then i think nothing is tried to be generated.

Post by Andreas PrieÃ

Post by James Carman
$ env | grep CATALINA
CATALINA_OPTS= -Xmx4096m -Xms2048m -XX:MaxPermSize=1024m
-XX:+UseParallelGC -server

Well, if you have increased memory for the permanent generation several
times and keep seeing this error, for me this sounds like a memory leak.
Tuning the java garbage collection will in all cases not help here,
because java only throws an out of memory error if garbage collection
does not help any more.
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/log/java/whatever
Then you can analyse what holds all the memory after the next crash.
http://www.eclipse.org/mat/
http://wiki.eclipse.org/index.php/MemoryAnalyzer
And you should search what they have at the tomcat pages and on their
http://wiki.apache.org/tomcat/OutOfMemory
This is most likely not specific to wicket itself.
Andreas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

Johan Compagner

2010-07-22 20:11:15 UTC

what kind of classes are serialized constantly?
are those proxies or other generated onces?

Somehow it has something to do with getting the default constructor of
the first none serializable class
And i think your first class in the hierarchy that is found that
doenst implement Serializable doesnt also have a default constructor
Then when that state is found it will generate a constructor method or
something (so some kind of quick asm wrapper class with that
constructor)

So look at the classes that you serialize, find the first one that is
none serializeble and give it a default constructor.
(it could be that that default constructor needs to be private)

johan

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

James Carman

2010-07-22 20:22:04 UTC

Post by Johan Compagner
what kind of classes are serialized constantly?
are those proxies or other generated onces?

All sorts of stuff I guess. This doesn't necessarily happen on one
particular page. It just starts happening after a while and the
application just starts throwing these stack traces.

Post by Johan Compagner
Somehow it has something to do with getting the default constructor of
the first none serializable class
And i think your first class in the hierarchy that is found that
doenst implement Serializable doesnt also have a default constructor
Then when that state is found it will generate a constructor method or
something (so some kind of quick asm wrapper class with that
constructor)
So look at the classes that you serialize, find the first one that is
none serializeble and give it a default constructor.
(it could be that that default constructor needs to be private)

Man, where to begin. I'm going to look at my IDE to see if it has a
code inspection for this! :)

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

Thomas Kappler

2010-07-23 11:24:02 UTC

Are you by chance using String.intern()? That is a candidate for filling
up the PermGen space.

As Andreas noted, the stack trace can be completely irrelevant to the
reason for running out of memory.

-- Thomas
--
-------------------------------------------------------------------
Thomas Kappler ***@isb-sib.ch
Swiss Institute of Bioinformatics Tel: +41 22 379 51 89
CMU, rue Michel Servet 1
1211 Geneve 4
Switzerland http://www.uniprot.org
-------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

James Carman

2010-07-23 11:26:32 UTC

I am not using intern() anywhere. Who knows if any of the gazillion
third-party libraries are using it, though. Time to fire up jmap

On Fri, Jul 23, 2010 at 7:24 AM, Thomas Kappler

Are you by chance using String.intern()? That is a candidate for filling up
the PermGen space.
As Andreas noted, the stack trace can be completely irrelevant to the reason
for running out of memory.
-- Thomas
--
-------------------------------------------------------------------
Swiss Institute of Bioinformatics Tel: +41 22 379 51 89
CMU, rue Michel Servet 1
1211 Geneve 4
Switzerland http://www.uniprot.org
-------------------------------------------------------------------
---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

Johan Compagner

2010-07-23 11:54:58 UTC

as far as i see now if i look that the stacktrace:

java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:413)
java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:310

so ObjectSTreamClass.lookup call there the constructor of ObjectStreamClass
lookup does check for caches but those are soft caches so these can be cleared.
It can mean 2 things, or somehow you have constantly different classes
(proxies or cglib generated other stuff?)
or the softcaches are cleared quite often
they kind of do that quite often, SoftReferences are NOT only cleared
when the jvm needs more mem but also much sooner!
You can set this system property: -XX:SoftRefLRUPolicyMSPerMB=3600000
that will result in a much longer live softreference...

Because there is constantly a ObjectStreamClass generated i think you
are then hitting the MethodAccessorGenerator:

private MagicAccessorImpl generate(final Class declaringClass,
String name,
Class[] parameterTypes,
Class returnType,
Class[] checkedExceptions,
int modifiers,
boolean isConstructor,
boolean forSerialization,
Class serializationTargetClass)

and that is constructing byte code on the fly and calling defineClass
on it which results in a new class all the time that is a
implementation of MagicAccessorImpl
that then is a "fake" constructor for your first none serializable
class (that didnt have one on its own)

I am not using intern() anywhere. Who knows if any of the gazillion
third-party libraries are using it, though. Time to fire up jmap
On Fri, Jul 23, 2010 at 7:24 AM, Thomas Kappler

Are you by chance using String.intern()? That is a candidate for filling up
the PermGen space.
As Andreas noted, the stack trace can be completely irrelevant to the reason
for running out of memory.
-- Thomas
--
-------------------------------------------------------------------
Swiss Institute of Bioinformatics Tel: +41 22 379 51 89
CMU, rue Michel Servet 1
1211 Geneve 4
Switzerland http://www.uniprot.org
-------------------------------------------------------------------
---------------------------------------------------------------------

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

James Carman

2010-07-23 12:12:28 UTC

Here's an interesting error message from jvisualvm:

*** Profiler engine warning: class
sun.reflect.GeneratedConstructorAccessor1 that should be instrumented
is not loaded by target VM
*** Requested classloader: ***@7dc5ddc9,
its class = class sun.reflect.DelegatingClassLoader, index = 542,
hashcode = 2110119369
*** Profiler engine warning: target VM cannot load class to instrument
sun.reflect.GeneratedConstructorAccessor1
*** probably it has been unloaded recently

Sounds like we're on the right track, perhaps?

Post by James Carman
java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:413)
java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:310
so ObjectSTreamClass.lookup call there the constructor of ObjectStreamClass
lookup does check for caches but those are soft caches so these can be cleared.
It can mean 2 things, or somehow you have constantly different classes
(proxies or cglib generated other stuff?)
or the softcaches are cleared quite often
they kind of do that quite often, SoftReferences are NOT only cleared
when the jvm needs more mem but also much sooner!
You can set this system property: -XX:SoftRefLRUPolicyMSPerMB=3600000
that will result in a much longer live softreference...
Because there is constantly a ObjectStreamClass generated i think you
private MagicAccessorImpl generate(final Class declaringClass,
String name,
Class[] parameterTypes,
Class returnType,
Class[] checkedExceptions,
int modifiers,
boolean isConstructor,
boolean forSerialization,
Class serializationTargetClass)
and that is constructing byte code on the fly and calling defineClass
on it which results in a new class all the time that is a
implementation of MagicAccessorImpl
that then is a "fake" constructor for your first none serializable
class (that didnt have one on its own)

I am not using intern() anywhere. Who knows if any of the gazillion
third-party libraries are using it, though. Time to fire up jmap
On Fri, Jul 23, 2010 at 7:24 AM, Thomas Kappler

Are you by chance using String.intern()? That is a candidate for filling up
the PermGen space.
As Andreas noted, the stack trace can be completely irrelevant to the reason
for running out of memory.
-- Thomas
--
-------------------------------------------------------------------
Swiss Institute of Bioinformatics Tel: +41 22 379 51 89
CMU, rue Michel Servet 1
1211 Geneve 4
Switzerland http://www.uniprot.org
-------------------------------------------------------------------
---------------------------------------------------------------------

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

Martijn Dashorst

2010-07-23 12:20:44 UTC

On Fri, Jul 23, 2010 at 2:12 PM, James Carman

Post by James Carman
*** Profiler engine warning: class
sun.reflect.GeneratedConstructorAccessor1 that should be instrumented
is not loaded by target VM
its class = class sun.reflect.DelegatingClassLoader, index = 542,
hashcode = 2110119369
*** Profiler engine warning: target VM cannot load class to instrument
sun.reflect.GeneratedConstructorAccessor1
*** probably it has been unloaded recently
Sounds like we're on the right track, perhaps?

In these kind of cases I've learnt to trust Johan.

Martijn

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

James Carman

2010-07-23 12:33:21 UTC

On Fri, Jul 23, 2010 at 8:20 AM, Martijn Dashorst

Post by Martijn Dashorst
In these kind of cases I've learnt to trust Johan.

It appears I'm learning that myself!

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

Johan Compagner

2010-07-23 12:37:59 UTC

yes those are the things that are generated:

private static synchronized String generateName(boolean isConstructor,
boolean forSerialization)
{
if (isConstructor) {
if (forSerialization) {
int num = ++serializationConstructorSymnum;
return
"sun/reflect/GeneratedSerializationConstructorAccessor" + num;
} else {
int num = ++constructorSymnum;
return "sun/reflect/GeneratedConstructorAccessor" + num;
}
} else {
int num = ++methodSymnum;
return "sun/reflect/GeneratedMethodAccessor" + num;
}
}

the only thing i dont get is that for you the forSerialization flag
should be true (at least the stack trace you shown us)
So for your serialization thing you should have:
GeneratedSerializationConstructorAccessor1 or something..

But i guess you have classes that generate those special constructors
even more also for not specific to serialization.

I am not using intern() anywhere. Who knows if any of the gazillion
third-party libraries are using it, though. Time to fire up jmap
On Fri, Jul 23, 2010 at 7:24 AM, Thomas Kappler

Are you by chance using String.intern()? That is a candidate for filling up
the PermGen space.
As Andreas noted, the stack trace can be completely irrelevant to the reason
for running out of memory.
-- Thomas
--
-------------------------------------------------------------------
Swiss Institute of Bioinformatics Tel: +41 22 379 51 89
CMU, rue Michel Servet 1
1211 Geneve 4
Switzerland http://www.uniprot.org
-------------------------------------------------------------------
---------------------------------------------------------------------

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

James Carman

2010-07-23 12:40:55 UTC

Post by Johan Compagner
the only thing i dont get is that for you the forSerialization flag
should be true (at least the stack trace you shown us)
GeneratedSerializationConstructorAccessor1 or something..

I have those too! :)

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

James Carman

2010-07-23 12:41:55 UTC

*** Profiler engine warning: class
sun.reflect.GeneratedSerializationConstructorAccessor394 that should
be instrumented is not loaded by target VM
*** Requested classloader: ***@51159ff2,
its class = class sun.reflect.DelegatingClassLoader, index = 582,
hashcode = 1360371698
*** Profiler engine warning: target VM cannot load class to instrument
sun.reflect.GeneratedSerializationConstructorAccessor394
*** probably it has been unloaded recently

On Fri, Jul 23, 2010 at 8:40 AM, James Carman

I have those too! :)

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

Johan Compagner

2010-07-23 12:55:42 UTC

ahh you are already at nr 394 of those :)

Let see how for they come :)

Post by James Carman
*** Profiler engine warning: class
sun.reflect.GeneratedSerializationConstructorAccessor394 that should
be instrumented is not loaded by target VM
its class = class sun.reflect.DelegatingClassLoader, index = 582,
hashcode = 1360371698
*** Profiler engine warning: target VM cannot load class to instrument
sun.reflect.GeneratedSerializationConstructorAccessor394
*** probably it has been unloaded recently
On Fri, Jul 23, 2010 at 8:40 AM, James Carman

I have those too! :)

---------------------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-***@wicket.apache.org
For additional commands, e-mail: users-***@wicket.apache.org

anshuiitk

2010-11-30 21:35:47 UTC