Skip to:

  1. Skip to Jira Navigation
  2. Skip to Side Navigation
  3. Skip to Main Content

Console error in session view when a cluster member goes offline

Description

Per email from Guus:

We quite often upgrade the domain at igniterealtime.org with a new Openfire build. Even when no cluster-related code has changed, we frequently get errors - notably on the admin panel page that displays the current user sessions. Stack traces will appear there, complaining about the unavailability of a particular cluster node identified by some byte array. The issue disappears after restarting both cluster nodes, while making sure that at some point in time all nodes are shut down.

Environment

None

Activity

Show:

Tom Evans January 9, 2013 at 10:31 PM

NOTE: Need to update both Openfire core and Hazelcast plugin (1.0.2) to pick up this fix.

Tom Evans January 9, 2013 at 10:29 PM

Per 13392: Improved exception handling and error recovery in core clustering classes; retooled cache configuration to prevent eviction of core components; updated versions and documentation for clustering and hazelcast plugins

Tom Evans January 9, 2013 at 10:18 PM

Initial fix covered the symptoms but not the root of the problem. Additional changes pending.

Tom Evans November 20, 2012 at 11:25 PM
Edited

Validate sessions before displaying in session UI. (SVN 13369/13370)

Tom Evans November 20, 2012 at 7:28 PM

This is an example of the error as reported:

java.lang.IllegalStateException: Requested node [B@3fa6b386 not found in cluster
at com.jivesoftware.util.cache.ClusteredCacheFactory.doSynchronousClusterTask(ClusteredCacheFactory.java:328)
at org.jivesoftware.util.cache.CacheFactory.doSynchronousClusterTask(CacheFactory.java:566)
at com.jivesoftware.openfire.session.RemoteSession.doSynchronousClusterTask(RemoteSession.java:173)
at com.jivesoftware.openfire.session.RemoteSession.isSecure(RemoteSession.java:131)
at org.jivesoftware.openfire.admin.session_002dsummary_jsp._jspService(session_002dsummary_jsp.java:362)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:547)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1359)
at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:118)
at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:52)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1330)
at org.jivesoftware.util.LocaleFilter.doFilter(LocaleFilter.java:74)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1330)
at org.jivesoftware.util.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:50)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1330)
at org.jivesoftware.admin.PluginFilter.doFilter(PluginFilter.java:78)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1330)
at org.jivesoftware.admin.AuthCheckFilter.doFilter(AuthCheckFilter.java:164)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1330)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:478)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:520)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:941)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:409)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:875)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:110)
at org.eclipse.jetty.server.Server.handle(Server.java:349)
at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:441)
at org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:919)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:582)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:218)
at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:51)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:586)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:44)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:598)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:533)
at java.lang.Thread.run(Thread.java:662)

This can be reproduced by loading the session page from a peer member after a cluster member has been shut down.

Fixed

Details

Assignee

Reporter

Labels

Components

Fix versions

Affects versions

Priority

Created November 19, 2012 at 11:56 PM
Updated January 9, 2013 at 10:31 PM
Resolved January 9, 2013 at 10:31 PM