Understanding Java Heap Space

Before going in depth , let us first understand the basics of heap and stack....

Heap - what exactly is this?

1. Class instances and arrays are stored in heap memory. Heap memory is also called as shared memory. As this is the place where multiple threads will share the same data .Instance variables and the Objects lie on Heap.
2. The heap is memory set aside for dynamic allocation. Unlike the stack, there's no enforced pattern to the allocation and deallocation of blocks from the heap, you can allocate/deallocate a block at any given time.
3. This includes the Objects created in local scope of any methods. In this case, reference variables to local objects are stored with method frame in a stack, but actual object lies in heap.
4. The heap is typically allocated at application startup by the runtime, and is reclaimed when the application (technically process) exits.

Stack - what exactly is this?

1. Local variables and methods lie on the Stack.
2. Java stacks (Sometimes referred as frames) are created private to a thread. Every thread will have a program counter (PC) and a java stack. PC will use the java stack to store the intermediate values, dynamic linking, return values for methods and dispatch exceptions. Every thread, including the main thread, daemons threads - get their own stack.
3. When a thread invokes a method, the JVM pushes a new frame onto that thread's Java stack.
4. All method calls, arguments, local variables, reference variables, intermediate computations and return values if any are kept in these stack corresponding to the method invoked.
5. The memory allocated for frame does not need to be contiguous.
6. The stack is always reserved in a LIFO order; the most recently reserved block is always the next block to be freed.
7. The stack is attached to a thread, so when the thread exits the stack is reclaimed.

Gist:

1. Local Variables are stored in stack during runtime.
2. Static Variables are stored in Method Area.
3. Arrays are stored in heap memory.
4. Stack does not need to be contiguous.
5. The stack is faster because the access pattern makes it trivial to allocate and deallocate memory from it (a pointer/integer is simply incremented or decremented), while the heap has much more complex bookkeeping involved in an allocation or free. Also, each byte in the stack tends to be reused very frequently which means it tends to be mapped to the processor's cache, making it very fast.

Heap Regions

[ Offline comment --> Used "sketchboard.me" provided by chrome store to make above snap. Cool stuff from Google. Try that out too :) ]

1. Eden Space (heap): pool from which memory is initially allocated for most objects.
2. Survivor Space (heap): pool containing objects that have survived GC of eden space.
3. Tenured Generation (heap): pool containing objects that have existed for some time in the survivor space.
4. Permanent Generation (non-heap): holds all the reflective data of the virtual machine itself, stores class level details, loading and unloading classes (e.g. JSPs), methods, String pool. PermGen contains meta-data of the classes and the objects i.e. pointers into the rest of the heap where the objects are allocated. The PermGen also contains Class-loaders which have to be manually destroyed at the end of their use else they stay in memory and also keep holding references to their objects on the heap.
5. Code Cache (non-heap): HotSpot JVM also includes a "code cache" containing     memory used for compilation and storage of native code.

Understanding Heap allocation and Garbage Collection

Garbage collection (GC) is how the JVM frees memory occupied by objects that are no longer referenced. Garbage collection is the process of releasing memory used by the dead objects. The algorithms and parameters used by GC can have dramatic effects on performance.

The Java HotSpot VM defines two generations: the young generation (sometimes called 
the "nursery") and the old generation. The young generation consists of an "Eden space"
and two "survivor spaces." The VM initially assigns all objects to the Eden space, and
most objects die there. When it performs a minor GC, the VM moves any remaining
objects from the Eden space to one of the survivor spaces. The VM moves objects that 
live long enough in the survivor spaces to the "tenured" space in the old generation. When
the tenured generation fills up, there is a full GC that is often much slower because it
involves all live objects. The permanent generation holds all the reflective data of the
virtual machine itself, such as class and method objects.

Figure: Generations of Data in Garbage Collection

Points to remember (Tips which can be very useful for tuning Heap):

Garbage collection can become a bottleneck in highly parallel systems.  By understanding how GC works, it is possible to use a variety of command line options to minimize that impact. Java heap allocation starts with min size -Xms and increases upto Xmx. At any point, it has used heap(heap actually in use), committed heap (allocated heap at that point. includes used + free), max heap(max heap that can be allocated).Try to keep -Xms and -Xmx same to reduce frequent Full GC.

The bigger the young generation, the less often minor collections occur. However, for a bounded heap size a larger young generation implies a smaller old generation, which will increase the frequency of major collections (full GC's)

-XX:NewRatio=3 means that the ratio between the young and old generation is 1:3; in other words, the combined size of eden and the survivor spaces will be one fourth of the heap.

Recommended JVM parameters (Can differ from application to application. Study more and get the best tuning for your application)

export CATALINA_OPTS="-Xmx4096m –Xms4096m -Xmn1g -XX:ParallelGCThreads= 16 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:SurvivorRatio=8  -XX:TargetSurvivorRatio=80 -XX:PermSize=512M -XX:MaxPermSize=1024M" Add -Xmn1g parameter [To mainly take care of young generation objects] Add  -XX:ParallelGCThreads=16  [Formula for this: We get 1 parallel GC thread per CPU for up to 8 CPUs, and 5/8 after that (so for 16 CPUs we get: 8 + 5/8 x 8 = 13  GC threads).] Add -XX:SurvivorRatio=8  Add -XX:TargetSurvivorRatio=80 -XX:NewRatio=3 Try to keep -Xms and -Xmx same to reduce frequent Full GC. Always use CATALINA_OPTS if if you're setting environment variables for used only by Tomcat, you'll be best advised to use CATALINA_OPTS, whereas if you're setting environment variables to be used by other java applications as well, such as by JBoss, you should put your settings in JAVA_OPTS. http://stackoverflow.com/questions/11222365/catalina-opts-vs-java-opts-what-is-the-difference

Use Jconsole to capture the clearest picture of how the different generations of memory are behaving for your application .Below is a good example of how to enable JMX port for monitoring your application using Jconsole/JvisualVM.

e.g: 

##### JConsloe options added by Kulshresht to analyse heap memory  ---------- CATALINA_OPTS="$CATALINA_OPTS  \                                -Dcom.sun.management.jmxremote \                                -Dcom.sun.management.jmxremote.port=15556 \                                -Dcom.sun.management.jmxremote.ssl=false \                                -Dcom.sun.management.jmxremote.authenticate=false\                                -Djava.rmi.server.hostname=19.83.73.82"

Jconsole & JvisualVM are inbuilt JDK tools. So there is no extra tension of installing other softwares (assuming you already have JDK installed :)). JvisualVM has great graphics. The only disadvantage with these tools is that they do not save historical data. So, if something goes wrong with your application at midnight then you can't trouble shoot the issue withe these graphs until and unless you have enabled the Jconsole UI (24*7) on a particular machine. Don't get disheartened , there is always a way :). The tool which can be used to store historical data is "Hyperic".

References:

http://docs.oracle.com/javase/1.5.0/docs/guide/management/jconsole.html
http://docs.oracle.com/javase/7/docs/technotes/guides/management/jconsole.html
http://stackoverflow.com/questions/79923/what-and-where-are-the-stack-and-heap

http://docs.oracle.com/cd/E19900-01/819-4742/abeik/index.html
http://docs.oracle.com/cd/E23095_01/Platform.93/ATGInstallGuide/html/s0606garbagecollection01.html
http://www.mastercorp.free.fr/Ing1/Cours/Java/java_lesson1/doc/Articles/GarbageCollection/tuning%20GC.htm
http://middlewaremagic.com/weblogic/?tag=eden-space
http://www.javaworld.com/javaworld/jw-01-2002/jw-0111-hotspotgc.html
http://middlewaremagic.com/weblogic/?p=4456
http://middlewaremagic.com/weblogic/?tag=eden-space
http://www.cubrid.org/blog/dev-platform/how-to-monitor-java-garbage-collection/

Article on PermGen: https://blogs.oracle.com/jonthecollector/entry/presenting_the_permanent_generation

Best tool to analyse thread dump

If we use jstack to take thread dump.Most of the time thread dump gets corrupted and it becomes difficult to analyse it.Tools like threadlogic, tda, can't parse a jstack thread dump, but there is a tool https://github.com/brass-kazoo/ParseJStack that produces a limited summary.

Try this out......https://github.com/brass-kazoo/ParseJStack

Courtsey : Peter Booth

Honest and Fake..........

I will share experiences of my life's journey ( though i had only completed 1/3rd of my life ....assuming i would live 85+ years ;) ) , in simple words, which may not leave you inspired, but will definitely help you to survive at least this life.But don't expect that i will show you the way to paradise :) . 

And if you follow my words then happiness (to some extent), creativity and success will follow on its own - or maybe not but you will have to live this life nevertheless. Don't come running after me if you are not able to achieve even a single one of them. But I assure you that you would applaud me at the end of this small speech and think that .....yeah this guy understand life not to the best of it but yaah to a better extent .

Journeys can be defined by age and time or even by destinations, as most often they are. But I feel it is hard for me to tell the story of my life in those terms because the concept of time has always eluded me. But i firmly believe that time is best medicine. It will not cure your pain completely but will work as anaesthesia and will keep you away from your past deeds ( categorically the bad ones ;)). Considering you are willing to allow time to inject anaesthesia in you . Remember, you have to move on in your life and never get stuck at any station. Life is like a train , don't get stuck at one station and don't let others to be the driver of your train.Don't get too much attach to anyone and don't let others to get over you. Everybody will leave you one day or other. Some will leave you because they had too and some without reason.Read this prayer somewhere and the writer couldn't have been more expressive "GOD grant me SERENITY to accept the things i cannot change,COURAGE to change the things I Can and WISDOM to know the difference."

Remember one thing "Change is the only constant thing in life" . Mark this golden words. It explains the whole life in 8 simple words.

Invest more in relationships.But don't expect returns.We human do that mistake every time and we have to pay the price at some stage.

It's always important to be in state of love to get best out of an individual. Love can be in any form.....You can be in love with your  Parents, Sister, Brother ,a girl, two girls ;) or a boy or any of your friend. It doesn't matter that you are in love with whom.........what matters is to be in state of love. I have been in love across all the category mentioned above (except the boy one :) .........though Gay marriage is now legal in many parts of world ;) ) . Jokes apart.........keep yourself in the state of love otherwise you would never know that whatever you are doing in life is for whom.You will be directionless/aimless. Understand the power of love. We youngster generally interpret love in wrong perspective. We think that loving a girl or rather beautiful girl is the only type of love which exist on this planet earth . In my case it was "Kristen Stewart" for a long span of life ......hahahahaha :).

Once my father told me when i was in standard four that "there can be substitute to your father but not your Mother." And he couldn't have been more expressive.Always respect your parents because they are the only one who will stand beside you till the end of your life.Parents want nothing in return, just respect their feeling, that's all.

Never ever try to imitate anybody. Be yourself. It's frustrating that I find myself living up to other people's interpretation of what I ought to be.It becomes a tight balance act, to keep doing what I do best and not be bothered by the reactions of people I do it for, in the first place.

Sometimes you would get hurt, feel tired, get frustrated but never give up.All I am is a fighter trying to balance my action and exterior reaction to my naked show of who I am inside. Learn to mock at yourselves too. Never become cynical about yourself. Becoming cynical about yourself or your life will destroy you at the end.

90% people don't care about your problem and rest 10% are happy to know that you have the problem (I will put myself in 90% category for most of the time....therefore it's not that I am untouched by these things.......But there are always 1% people for whom you matter everything and they matter everything for you. These 1% can be your family members, friends etc. But this 1% doesn't come under the same 100% mentioned earlier because they are beyond that 100% but are only 1%. I assume you got what i meant ;) ). So best way is don't discuss your problems with anyone. It's bullshit that people say that you will ease yourself by talking your problems with others. It's a myth. Face the issue/situation on your own because you are the originator of that problem and no one else, so don't expect others to solve your problem. You are alone doesn't mean that you are weak, It means that you are strong enough to handle all problems alone.

Success is a wonderful thing but we don't acquire wisdom from it. Therefore enjoy it ,feel it but don't try to push it hard on others. Will explain this in detail some other time in my next blog ( This space seems to be too small :) ). I had read "You can win" by shiv khera when i was in 12th standard and got inspired for 2-3 days but the tempo went down after couple of days. Motivation is a good thing but it needs to come from inside and not by someone else. I have always been terrified of failure since my childhood and that's the reason which kept me going. I am not trying to be pessimistic here but this is the truth of life,believe it or not.

So my friends, I am giving rest to my fingers here. I know that I have been contradictory to myself at many places but that's what I am.......Honest and Fake.

How to Remove unnecessary white spaces from xhtml/xml/html content - Mod_Trim

Want to remove unnecessary white spaces from xhtml/xml/html content? Which Makes your page size reduced by 25-30 % and enhance your application performance.

Detail:

Mod_Trim is an apache module which removes unnecessary white spaces from xhtml/xml/html content

Steps to implement this:

1) Go to website http://www.thrull.com/corner/webserver/mod-trim/ and download mod_trim module.

2) Run below commands. Remember to have sudo access on your server

sudo yum install httpd-devel [For apxs]
which apxs
./configure --with-apxs=/usr/sbin/apxs
cd src
make -f Makefile.apxs

[kulshresht@server123 .libs]$ pwd
/tmp/mod_trim-1.0/src/.libs
[kulshresht@server123 .libs]$ sudo cp -r mod_trim.so /etc/httpd/modules


2) Uncommnet this in httpd.conf file of Apache.
LoadModule filter_module modules/mod_filter.so


3)  Create a file trim.conf in conf.d folder of Apache and put below lines in that file :
# enable module (this line should sometimes be added in modules.conf if you have one)
LoadModule trim_module modules/mod_trim.so

# add filtering
FilterProvider trim-filter TRIM resp=Content-Type $text/html
FilterChain trim-filter

4) Restart Apache and you are done. 

Explained better at this link: http://www.thrull.com/corner/webserver/mod-trim/

Note :

One most important point to remember in the course of making these changes is:

Do not use one line JavaScript (e.g. // … double slash) instead of that use block comment (e.g /*.......*/)

Usage of single line comment might treat your entire code block as comment because mod_trim will trim new lines and or spaces. See below code snippet for explanation:

Before mod_trim action

// Add a script element as a child of the loadAnalyticsScript
<!--  GOOGLE ANALYSTICS SCRIPT STARTS -->
          var _gaq = _gaq || [];
          _gaq.push(['_setAccount', 'UA-2260765-1']);
...


After mod_trim action

// Add a script element as a child of the loadAnalyticsScript <!--  GOOGLE ANALYSTICS SCRIPT STARTS --> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-2260765-1']);

How to install Memcache from source code

How to install Memcache from source code

There are two ways to install memcache :

1. From some repository (CentOS,RHEL etc). [e.g: yum install memcached]
2. From source code –> I would prefer this method because sometimes the CentOS or RHEL repository   doesn't have the latest version . I had personally faced this issue.

To cross check and verify that memcahe is installed and running fine. Try below command:

#netstat -nap |grep LISTEN |grep 11211

(No info could be read for "-p": geteuid()=507 but you should be root.)

tcp        0      0 0.0.0.0:11211               0.0.0.0:*                   LISTEN      -

tcp        0      0 :::11211                    :::*                        LISTEN      -

Note: If you are installing using source code the Libevent would be required. Install that using Yum command. http://libevent.org/

As I write the latest Memcahed version is 1.4.15 . Pick up the latest version from below link: https://code.google.com/p/memcached/wiki/ReleaseNotes

Start memcahe from command prompt :
Command : nohup /usr/bin/memcached -p 11211 -u memcached -m 4096 -vv -I 10 &

Alternative way : /etc/init.d/memcached start/stop

It's preferred to install memcache in /opt/ directory, if you are installing memcache from source code. So that you don't overwrite other version of any other software running on the machine.

Steps to install memcache from source code:

# wget http://memcached.googlecode.com/files/memcached-1.4.15.tar.gz
# wget https://github.com/downloads/libevent/libevent/libevent-2.0.21-stable.tar.gz
# tar -xzf memcached-1.4.15.tar.gz
# tar -xzf libevent-2.0.21-stable.tar.gz
# tar -xzf memcache-3.0.6.tgz

# cd libevent-2.0.21-stable [ you can get it from http://libevent.org/ ]
# ./configure --prefix=/usr --libdir=/usr/lib64
# make
# make install

# cd ..

# cd memcached-1.4.15
# ./configure --prefix=/usr --libdir=/usr/lib64
# make
# make install

Classes, sizes, and chunks are shown best by starting up memcached with -vv:

[kulshresht@server123 bin]$ ./memcached -vv
slab class   1: chunk size        96 perslab   10922
slab class   2: chunk size       120 perslab    8738
slab class   3: chunk size       152 perslab    6898
slab class   4: chunk size       192 perslab    5461
slab class   5: chunk size       240 perslab    4369
slab class   6: chunk size       304 perslab    3449
slab class   7: chunk size       384 perslab    2730
slab class   8: chunk size       480 perslab    2184
slab class   9: chunk size       600 perslab    1747
slab class  10: chunk size       752 perslab    1394
slab class  11: chunk size       944 perslab    1110

Please feel free to drop me a mail if you are facing any issue(s) with installation.

If you want to explore further about how to monitor and debug memcache, please got to below link:
http://kulshresht-gautam.blogspot.in/2013/08/how-to-monitor-and-debug-memcached.html

References:

http://www.tecmint.com/install-memcached-caching-server-on-rhel-centos-fedora/

http://docs.oracle.com/cd/E17952_01/refman-5.0-en/ha-memcached-install.html

How to Integrate tomcat with memcahed for session clustering

Integrating tomcat with memcached for session clustering

Detail:https://code.google.com/p/memcached-session-manager/wiki/SetupAndConfiguration

It’s bit confusing there. Will make it simple for you.

Download below four jars to your tomcat lib folder:

     1)  memcached-session-manager-1.6.4.jar

     2)   memcached-session-manager-tc7-1.6.4.jar

     3)  spymemcached-2.8.12.jar

     4)  couchbase-client-1.1.4.jar

And add below configuration in your context.xml file located inside /conf folder of tomcat.

<!--    <Valve className="org.apache.catalina.valves.CometConnectionManagerValve" />   -->  

     <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"

                 memcachedNodes="n1:web1.prod:11211,n2:web2.prod:11211"

                 failoverNodes="n2"

                 requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"

        />

Restart your application and you are done with your tomcat-memcache integration. Wasn’t it quiet simple :)....

Few things to remember:

Tomcat-1 (t1) will primarily store it's sessions in memcached-2 (m2) which is running on another machine (m2 is a regular node for t1). Only if m2 is not available, t1 will store it's sessions in memcached-1 (m1, m1 is the failoverNode for t1). With this configuration, sessions won't be lost when machine 1 (serving t1 and m1) crashes.

<t1>   <t2>
  . \ / .
  .  X  .
  . / \ .
<m1>   <m2>

The default strategy uses java serialization and is already provided by the memcached-session-manager jar.

SERVER_ERROR object too large for cache

SERVER_ERROR object too large for cache

> I get "SERVER_ERROR object too large for cache" when reading a large 

> value from memcached.

Solution: Start memcache with -I option.

/opt/bin/memcached -p 11211 -u memcached -m 4096 -vv -I 10 &

Now memcache provides Configurable maximum item size

Many people have asked for memcached to be able to store items larger than 1MB, while it's generally

recommended that one not do this, it is now supported on the commandline. A few enlightened folk have also asked for memcached to reduce the maximum item size. That is also an option.

The new -I parameter allows you to specify the maximum item size at runtime. It supports a unit
postfix to allow for natural expression of item size.

Examples:

memcached -I 256k # Refuse items larger than 256k.


memcached -I 5m  # Allow objects up to 5MB

Memcached used as session replication/manager

What is memcache?

Memcache is an in-memory, distributed cache. The primary API for interacting with it are SET(key, value) and GET(key) operations. Memcache is essentially a hashmap (or dictionary) that is spread across multiple servers, where operations are still performed in constant time.

The most common usage of memcache is to cache expensive database queries and HTML renders such that these expensive operations don’t need to happen over and over again.

But in this post I would be focusing on how can we use memcache for session-replication.

How is memcache so fast?

Memcache is so fast for two reasons: the cache is entirely in-memory; and operations are performed in constant time. Memcache avoids writing any data to disk, which results in faster operations and no fault tolerance. If the server running memcache is restarted, the keys on that server will be lost, which is fine for the use case of memcache, as a temporal caching layer and not a definitive data store. The use of constant time operations is simply a deployment of all those good old tricks and tips for a computer science algorithms & data structures class. O(1) is a nice property to have.

Eviction, Expiration, and Memory Limits

Besides getting and setting keys, eviction and expiration are major components of memcache, too. When a key is SET in the cache, an expiration is set along with the key. Most memcache clients have a default expiration which can be optionally overwritten on a per-key basis.

Memcache doesn’t evict keys when their expiration time expires, as doing so isn’t possible while still guaranteeing O(1) operation times. Instead expiration is more of a way to say how long until a key should be considered stale. When a GET is performed, Memcache checks if the key’s expiration time is still valid before returning it.

A key is evicted from the cache when the total cache size has reached the limit. Most memcache implementations offer a least recently used (LRU) eviction policy, which evicts the key that was least recently used when a new key is added to the cache and the cache has reached its limit.

In general, the higher your cache limit, the fewer evictions you’ll have, and the higher your hit-rate will be, ultimately resulting in better performance and scalability.

Eviction meaning in simple terms: Removes items from the cache to free memory for new items.

Note: A key/value pair stored in memcached can get evicted prior to its expiry if there is still free space available?

Memcache allocates space in chuncks vs on-demand, and then stores items into the chunks and manages that memory manually. As a result, smaller items can "use" much larger pieces of memory than they would if space was allocated on a per-item basis.

To see stats from server side . A quick way to get memcache status

[kulshresht@web06 bin]$ echo stats | nc 172.16.0.52 11211 STAT pid 6274 STAT uptime 1807302 STAT time 1373449782 STAT version 1.4.15 STAT libevent 2.0.21-stable STAT pointer_size 64 STAT rusage_user 3916.447609 STAT rusage_system 7673.810403 STAT curr_connections 28 STAT total_connections 7126 STAT connection_structures 31 STAT reserved_fds 20 STAT cmd_get 60623944 STAT cmd_set 65457134 STAT cmd_flush 0 STAT cmd_touch 0 STAT get_hits 27331 STAT get_misses 60596613 STAT delete_misses 25500664 STAT delete_hits 224497 STAT incr_misses 0 STAT incr_hits 0 STAT decr_misses 0

.

.

.

.

[kulshresht@server1 bin]$ telnet 127.0.0.1 11211 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. stats STAT pid 6274 STAT uptime 1808371 STAT time 1373450851 STAT version 1.4.15 STAT libevent 2.0.21-stable STAT pointer_size 64 STAT rusage_user 3919.035216 STAT rusage_system 7677.627823 STAT curr_connections 28 STAT total_connections 7131 STAT connection_structures 31

To get the beautful UI for memcahe go to below URL:

https://code.google.com/p/phpmemcacheadmin/

It's worth noting that memcached uses it's own slab allocator instead of using standard malloc()/free() in order to be speed up memory management.

Memcached allocates memory in blocks of 1MB (by default), referred to as a slab. Slabs belong to a slab class, which is defined by the size of item that is stored in that slab.Lets say that we have two slab classes, a slab class of 1KB and the 2nd class at 2.25KB (the default growth factor in slab classes is 1.25 ). An item of 500bytes or anything less than 1KB would get put into slab of 1KB slab class and any item  >1KB and <2.25KB will go in 2nd slab of 2.25KB.

Memcahe will make slab on it's own depending on need and demand. Just keep in mind to give -I parameter at the time of memcache startup. Otherwise you will get "SERVER_ERROR object too large for cache" if cache exceeds the maximum slab size which is 1 MB by default.

memcached -I 10m  # Allow objects up to 10MB

For my application memcache made 52 slabs and max chunk size was of 10MB.

We can refuse items larger than particular size too. e.g: memcached -I 128k # Refuse items larger than 128k.

When an itme is stored then first there is a check to see which class it should fall into , based on it's size, and then memcached will check that class's stack to see if there are any empty spots available. If not, a new slab will be allocated to store the new item.

All this leads to one conclusion, that there is some tradeoff in unused memory. Not every item is going to be exactly the size defined by the slab class, or you could have just one item stored in a class for items of size 1k, meaning that 1MB is allocated just to store that 1k item. But these operations are very fast and very simple.