<div dir="ltr">That makes sense and the script is writing itself! It seems that jmap and jhat would be good to: <a href="http://www.marcsturlese.com/2009/05/09/analyzing-java-heaps-with-jmap-and-jhat/">http://www.marcsturlese.com/2009/05/09/analyzing-java-heaps-with-jmap-and-jhat/</a><div>
<br></div><div>To get jvisualvm working, I'm assuming I need to put Gnome on the server and then use putty with X-windows and forwarding enabled? If this is correct, are there performance ramifications to adding Gnome? If not, any references to how to do this?</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Oct 9, 2013 at 4:19 PM, James Mcphee <span dir="ltr"><<a href="mailto:jmcphe@gmail.com" target="_blank">jmcphe@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">You can plot a graph during these times to see if it's spending a lot of time in garbage collection. Other options are to take jstack thread dumps and see what the app is up to at that time.<div>
<br></div>
<div>My favorite way to fiddle is to plug VisualVM (replacement for jconsole) into the jvm and watch when issues are occuring. That uses JMX and you can monitor threads and mbeans.</div></div><div class="HOEnZb"><div class="h5">
<div class="gmail_extra"><br>
<br><div class="gmail_quote">On Wed, Oct 9, 2013 at 4:13 PM, Vimal Shah <span dir="ltr"><<a href="mailto:vimals@sokikom.com" target="_blank">vimals@sokikom.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">jstat works nicely now James. Thank you. I will try the various options as well.<div><br></div><div>A little more about the problem. Every 4-6 weeks the CPU util looks like:</div><div><br></div><div><img src="cid:ii_1419f75cdc403b65" alt="Inline image 1"><br>
</div><div><br></div><div><br></div><div>The application slows and Alerts are emailed. The application is restarted (in this case on 10/2 at 1:00). This allows the application to return to normal responsiveness and allows the box itself to go back to 3% CPU util. The ideal tool would be one that would help us understand this. Can jstat be used to investigate this? Would a call trace and heap dump be more useful as identification to the function that was called when this error occurred?</div>
<div><br></div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Oct 9, 2013 at 3:25 PM, James Mcphee <span dir="ltr"><<a href="mailto:jmcphe@gmail.com" target="_blank">jmcphe@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">For jstat, you have to be a user with permission to the process. In this case, either the user the java process is running as, or root. I use jstat when I want to script things to watch them on the console.<div>
<br></div><div>For JMX, there are jvm arguments you can use to open it up to external hosts, provide authentication, etc. That will depend on the server you're running and you should consult the documentation for that server.</div>
</div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Oct 9, 2013 at 3:19 PM, Vimal Shah <span dir="ltr"><<a href="mailto:vimals@sokikom.com" target="_blank">vimals@sokikom.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">James,<div><br></div><div>What other details can I provide?</div><div><br></div><div>It does sound like JMX is the standard. When running jstat with my JAVA PID </div>
<div><br></div><div><div>$ jstat -gc 19326</div>
<div>Could not attach to 19326</div></div><div><br></div><div>I don't have JMX configured. After some reading it seems that I need to get a Mbeans server running. Is this what I have to do?</div><div><br></div><div><a href="http://docs.oracle.com/javase/tutorial/jmx/mbeans/standard.html" target="_blank">http://docs.oracle.com/javase/tutorial/jmx/mbeans/standard.html</a><br>
</div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Oct 9, 2013 at 1:15 PM, James Mcphee <span dir="ltr"><<a href="mailto:jmcphe@gmail.com" target="_blank">jmcphe@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">We'd need more details to give you much more than "Using JMX or jstat would be easiest"<div>
<br></div><div>Example:</div><div><div>./jstat -gc 27104</div><div> S0C S1C S0U S1U EC EU OC OU PC PU YGC YGCT FGC FGCT GCT </div>
<div>20928.0 21312.0 0.0 4864.0 305728.0 128383.7 262656.0 223537.3 170240.0 130738.6 295 6.912 10 8.268 15.180</div></div><div><br></div><div><a href="http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstat.html" target="_blank">http://docs.oracle.com/javase/1.5.0/docs/tooldocs/share/jstat.html</a><br>
</div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Wed, Oct 9, 2013 at 1:05 PM, Vimal Shah <span dir="ltr"><<a href="mailto:vimals@sokikom.com" target="_blank">vimals@sokikom.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">Hello,<div><br></div><div>Does anyone have a step by step guide to getting monitoring (heap stack) set up of a Java app on an Amazon VPC box? The application is smartfoxserver and it's on Ubuntu 12.04.</div>
<div><br></div><div>I've scratched the surface on the following:<br clear="all"><div><ul><li><a href="https://code.google.com/p/javamelody/" target="_blank">https://code.google.com/p/javamelody/</a><br></li><li><a href="http://docs.oracle.com/javase/6/docs/technotes/guides/management/jconsole.html" target="_blank">http://docs.oracle.com/javase/6/docs/technotes/guides/management/jconsole.html</a><br>
</li><li><a href="http://jamonapi.sourceforge.net/" target="_blank">http://jamonapi.sourceforge.net/</a><br></li></ul><div><br></div></div><div>Application code changes aren't within my reach at the moment. An apt-get install of a monitoring tool on the server under question would be ideal. If this tool can have low overhead even better.</div>
<div><br></div><div>Thank you all in advance.</div><span><font color="#888888"><div><br></div>-- <br><font size="3"><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><font face="arial, helvetica, sans-serif">Vimal (rhymes with Kimmel) Shah</font></div>
<div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,helvetica,sans-serif;font-size:13px">Front-End / Infrastructure Engineer</span></div><span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><font face="arial, helvetica, sans-serif">Sokikom<br>
Mobile: <a href="tel:%28480%29%20752-9269" value="+14807529269" target="_blank">(480) 752-9269</a><br>Email:<font color="#500050"> </font><a href="mailto:vimals@sokikom.com" style="color:rgb(42,93,176)" target="_blank">vimals@sokikom.com</a></font></span><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">
<font face="arial, helvetica, sans-serif">Web:<font color="#500050"> </font><font color="#3333FF" style="color:rgb(42,93,176)"><a href="http://www.sokikom.com/" style="color:rgb(42,93,176)" target="_blank">www.sokikom.com</a></font></font></div>
<div><font face="arial, helvetica, sans-serif"><br></font></div></font><div><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">Follow us: <a href="http://www.twitter.com/sokikom" style="color:rgb(42,93,176)" target="_blank">twitter.com/sokikom</a></div>
<div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">Like us: <a href="http://www.facebook.com/sokikom" style="color:rgb(42,93,176)" target="_blank">facebook.com/sokikom</a></div></div>
</font></span></div></div>
<br></div></div>---------------------------------------------------<br>
PLUG-discuss mailing list - <a href="mailto:PLUG-discuss@lists.phxlinux.org" target="_blank">PLUG-discuss@lists.phxlinux.org</a><br>
To subscribe, unsubscribe, or to change your mail settings:<br>
<a href="http://lists.phxlinux.org/mailman/listinfo/plug-discuss" target="_blank">http://lists.phxlinux.org/mailman/listinfo/plug-discuss</a><span><font color="#888888"><br></font></span></blockquote></div>
<span><font color="#888888"><br><br clear="all"><div><br></div>-- <br>James McPhee<br>
<a href="mailto:jmcphe@gmail.com" target="_blank">jmcphe@gmail.com</a>
</font></span></div>
<br>---------------------------------------------------<br>
PLUG-discuss mailing list - <a href="mailto:PLUG-discuss@lists.phxlinux.org" target="_blank">PLUG-discuss@lists.phxlinux.org</a><br>
To subscribe, unsubscribe, or to change your mail settings:<br>
<a href="http://lists.phxlinux.org/mailman/listinfo/plug-discuss" target="_blank">http://lists.phxlinux.org/mailman/listinfo/plug-discuss</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br><font size="3"><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">
<font face="arial, helvetica, sans-serif">Vimal (rhymes with Kimmel) Shah</font></div><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,helvetica,sans-serif;font-size:13px">Front-End / Infrastructure Engineer</span></div>
<span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><font face="arial, helvetica, sans-serif">Sokikom<br>Mobile: <a href="tel:%28480%29%20752-9269" value="+14807529269" target="_blank">(480) 752-9269</a><br>
Email:<font color="#500050"> </font><a href="mailto:vimals@sokikom.com" style="color:rgb(42,93,176)" target="_blank">vimals@sokikom.com</a></font></span><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">
<font face="arial, helvetica, sans-serif">Web:<font color="#500050"> </font><font color="#3333FF" style="color:rgb(42,93,176)"><a href="http://www.sokikom.com/" style="color:rgb(42,93,176)" target="_blank">www.sokikom.com</a></font></font></div>
<div><font face="arial, helvetica, sans-serif"><br></font></div></font><div><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">Follow us: <a href="http://www.twitter.com/sokikom" style="color:rgb(42,93,176)" target="_blank">twitter.com/sokikom</a></div>
<div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">Like us: <a href="http://www.facebook.com/sokikom" style="color:rgb(42,93,176)" target="_blank">facebook.com/sokikom</a></div></div>
</div>
</div></div><br>---------------------------------------------------<br>
PLUG-discuss mailing list - <a href="mailto:PLUG-discuss@lists.phxlinux.org" target="_blank">PLUG-discuss@lists.phxlinux.org</a><br>
To subscribe, unsubscribe, or to change your mail settings:<br>
<a href="http://lists.phxlinux.org/mailman/listinfo/plug-discuss" target="_blank">http://lists.phxlinux.org/mailman/listinfo/plug-discuss</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br>James McPhee<br>
<a href="mailto:jmcphe@gmail.com" target="_blank">jmcphe@gmail.com</a>
</div>
</div></div><br>---------------------------------------------------<br>
PLUG-discuss mailing list - <a href="mailto:PLUG-discuss@lists.phxlinux.org" target="_blank">PLUG-discuss@lists.phxlinux.org</a><br>
To subscribe, unsubscribe, or to change your mail settings:<br>
<a href="http://lists.phxlinux.org/mailman/listinfo/plug-discuss" target="_blank">http://lists.phxlinux.org/mailman/listinfo/plug-discuss</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br><font size="3"><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">
<font face="arial, helvetica, sans-serif">Vimal (rhymes with Kimmel) Shah</font></div><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,helvetica,sans-serif;font-size:13px">Front-End / Infrastructure Engineer</span></div>
<span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><font face="arial, helvetica, sans-serif">Sokikom<br>Mobile: <a href="tel:%28480%29%20752-9269" value="+14807529269" target="_blank">(480) 752-9269</a><br>
Email:<font color="#500050"> </font><a href="mailto:vimals@sokikom.com" style="color:rgb(42,93,176)" target="_blank">vimals@sokikom.com</a></font></span><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">
<font face="arial, helvetica, sans-serif">Web:<font color="#500050"> </font><font color="#3333FF" style="color:rgb(42,93,176)"><a href="http://www.sokikom.com/" style="color:rgb(42,93,176)" target="_blank">www.sokikom.com</a></font></font></div>
<div><font face="arial, helvetica, sans-serif"><br></font></div></font><div><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">Follow us: <a href="http://www.twitter.com/sokikom" style="color:rgb(42,93,176)" target="_blank">twitter.com/sokikom</a></div>
<div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">Like us: <a href="http://www.facebook.com/sokikom" style="color:rgb(42,93,176)" target="_blank">facebook.com/sokikom</a></div></div>
</div>
</div></div><br>---------------------------------------------------<br>
PLUG-discuss mailing list - <a href="mailto:PLUG-discuss@lists.phxlinux.org" target="_blank">PLUG-discuss@lists.phxlinux.org</a><br>
To subscribe, unsubscribe, or to change your mail settings:<br>
<a href="http://lists.phxlinux.org/mailman/listinfo/plug-discuss" target="_blank">http://lists.phxlinux.org/mailman/listinfo/plug-discuss</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br>James McPhee<br>
<a href="mailto:jmcphe@gmail.com" target="_blank">jmcphe@gmail.com</a>
</div>
</div></div><br>---------------------------------------------------<br>
PLUG-discuss mailing list - <a href="mailto:PLUG-discuss@lists.phxlinux.org">PLUG-discuss@lists.phxlinux.org</a><br>
To subscribe, unsubscribe, or to change your mail settings:<br>
<a href="http://lists.phxlinux.org/mailman/listinfo/plug-discuss" target="_blank">http://lists.phxlinux.org/mailman/listinfo/plug-discuss</a><br></blockquote></div><br><br clear="all"><div><br></div>-- <br><font size="3"><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">
<font face="arial, helvetica, sans-serif">Vimal (rhymes with Kimmel) Shah</font></div><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,helvetica,sans-serif;font-size:13px">Front-End / Infrastructure Engineer</span></div>
<span style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px"><font face="arial, helvetica, sans-serif">Sokikom<br>Mobile: (480) 752-9269<br>Email:<font color="#500050"> </font><a href="mailto:vimals@sokikom.com" style="color:rgb(42,93,176)" target="_blank">vimals@sokikom.com</a></font></span><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">
<font face="arial, helvetica, sans-serif">Web:<font color="#500050"> </font><font color="#3333FF" style="color:rgb(42,93,176)"><a href="http://www.sokikom.com/" style="color:rgb(42,93,176)" target="_blank">www.sokikom.com</a></font></font></div>
<div><font face="arial, helvetica, sans-serif"><br></font></div></font><div><div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">Follow us: <a href="http://www.twitter.com/sokikom" style="color:rgb(42,93,176)" target="_blank">twitter.com/sokikom</a></div>
<div style="border-collapse:collapse;font-family:arial,sans-serif;font-size:13px">Like us: <a href="http://www.facebook.com/sokikom" style="color:rgb(42,93,176)" target="_blank">facebook.com/sokikom</a></div></div>
</div>