Version: 9.4.5.v20170502 |
private support for your internal/customer projects ... custom extensions and distributions ... versioned snapshots for indefinite support ... scalability guidance for your apps and Ajax/Comet projects ... development services for sponsored feature development
The Java Management Extensions (JMX) API is a standard API for managing and monitoring resources such as applications, devices, services, and the Java virtual machine.
Typical uses of the JMX technology include:
The JMX API includes remote access, so a remote management program can interact with a running application for these purposes.
Jetty’s architecture is based on POJO components (see Jetty Architecture).
These components are organized in a tree and each component may have a lifecycle
that spans the Server
lifetime, or a web application lifetime, or even shorter
lifetimes such as that of a TCP connection.
Every time a component is added or removed from the component tree, an event is
emitted, and Container.Listener
implementations can listen to those events and perform additional actions.
One such Container.Listener
is MBeanContainer
that uses ObjectMBean
to
create an MBean from an arbitrary POJO, and register/unregister the MBean to/from
the platform MBeanServer
.
Jetty components are annotated with JMX annotations
and provide specific JMX details so that ObjectMBean
can build a more
precise representation of the JMX metadata associated with the component POJO.
Therefore, when a component is added to the component tree, MBeanContainer
is
notified, it creates the MBean from the component POJO and registers it to
the MBeanServer
.
Similarly, when a component is removed from the tree, MBeanContainer
is
notified, and unregisters the MBean from the MBeanServer
.
The Jetty MBeans can be accessed via any JMX console such as Java Mission Control (JMC), VisualVM, JConsole or others.
This guide describes the various ways to initialize and configure the Jetty JMX integration.
Configuring the Jetty JMX integration only registers the Jetty MBeans into the platform
MBeanServer
, and therefore the MBeans can only be accessed locally (from the same machine),
not from remote machines.
This means that this configuration is enough for development, where you have easy access (with graphical user interface) to the machine where Jetty runs, but it is typically not enough when the machine Jetty where runs is remote, or only accessible via SSH or otherwise without graphical user interface support. In these cases, you have to enable JMX remote access.
JMX is not enabled by default in the Jetty distribution.
To enable JMX in the Jetty distribution run the following, where {$jetty.home}
is the directory where you have the Jetty distribution installed, and
${jetty.base}
is the directory where you have your Jetty configuration
(see the documentation for Jetty base vs. home examples):
$ cd ${jetty.base} $ java -jar {$jetty.home}/start.jar --add-to-start=jmx
Running the above command will append the available configurable elements of the jmx
module
to the {$jetty.base}/start.ini
file, or create the ${jetty.base}/start.d/jmx.ini
file.
When running Jetty embedded into an application, create and configure an MBeanContainer
instance as follows:
Server server = new Server();
// Setup JMX.
MBeanContainer mbeanContainer = new MBeanContainer(ManagementFactory.getPlatformMBeanServer());
server.addBean(mbeanContainer);
// Export the loggers as MBeans.
server.addBean(Log.getLog());
Because logging is initialized prior to the MBeanContainer
(even before the Server
itself),
it is necessary to register the logger manually via server.addBean()
so that the loggers may
show up in the JMX tree as MBeans.
If you are using the Jetty Maven plugin you should copy the
${jetty.home}/etc/jetty-jmx.xml
file into your webapp project somewhere, such as
src/main/config/etc/
, then add a
<jettyXml>
element to the <configuration>
element of the Jetty Maven Plugin:
<plugin>
<groupid>org.eclipse.jetty</groupid>
<artifactid>jetty-maven-plugin</artifactid>
<version>9.4.5.v20170502</version>
<configuration>
<scanintervalseconds>10</scanintervalseconds>
<jettyXml>src/main/config/etc/jetty-jmx.xml</jettyXml>
</configuration>
</plugin>
The simplest way to access the MBeans that Jetty publishes is to use Java Mission Control (JMC) or JConsole.
Both these tools can connect to local or remote JVMs to display the MBeans.
For local access, you just need to start JConsole or JMC and then choose from their user interface the local JVM you want to connect to.
For remote access, you need first to enable JMX remote access in Jetty.
There are two ways of enabling remote connectivity so that JConsole or JMC can connect to the remote JVM to visualize MBeans.
com.sun.management.jmxremote
system property on the command line.
Unfortunately, this solution does not work well with firewalls and is not flexible.jmx-remote
module or - equivalently - the ConnectorServer
class.ConnectorServer
will use by default RMI to allow connection from remote clients,
and it is a wrapper around the standard JDK class JMXConnectorServer
, which is
the class that provides remote access to JMX clients.
Connecting to the remote JVM is a two step process:
JMXConnectorServer
; this RMI stub contains the IP address and port to connect
to the RMI server, i.e. the remote JMXConnectorServer
.JMXConnectorServer
) typically on an address and port that may be different
from the RMI registry address and port.The configuration for the RMI registry and the RMI server is specified by a JMXServiceURL
.
The string format of an RMI JMXServiceURL
is:
service:jmx:rmi://<rmi_server_host>:<rmi_server_port>/jndi/rmi://<rmi_registry_host>:<rmi_registry_port>/jmxrmi
Default values are:
rmi_server_host = localhost rmi_server_port = 1099 rmi_registry_host = localhost rmi_registry_port = 1099
With the default configuration, only clients that are local to the server machine can connect to the RMI registry and RMI server - this is done for security reasons. With this configuration it would still be possible to access the MBeans from remote using a SSH tunnel.
By specifying an appropriate JMXServiceURL
, you can fine tune the network interfaces the
RMI registry and the RMI server bind to, and the ports that the RMI registry and the RMI server
listen to.
The RMI server and RMI registry hosts and ports can be the same (as in the default configuration)
because RMI is able to multiplex traffic arriving to a port to multiple RMI objects.
If you need to allow JMX remote access through a firewall, you must open both the RMI registry and the RMI server ports.
Examples:
service:jmx:rmi:///jndi/rmi:///jmxrmi rmi_server_host = local host address rmi_server_port = randomly chosen rmi_registry_host = local host address rmi_registry_port = 1099 service:jmx:rmi://0.0.0.0:1099/jndi/rmi://0.0.0.0:1099/jmxrmi rmi_server_host = any address rmi_server_port = 1099 rmi_registry_host = any address rmi_registry_port = 1099 service:jmx:rmi://localhost:1100/jndi/rmi://localhost:1099/jmxrmi rmi_server_host = loopback address rmi_server_port = 1100 rmi_registry_host = loopback address rmi_registry_port = 1099
When ConnectorServer
is started, its RMI stub is exported to the RMI registry.
The RMI stub contains the IP address and port to connect to the RMI object, but
the IP address is typically the machine host name, not the host specified in the
JMXServiceURL
.
To control the IP address stored in the RMI stub you need to set the system
property java.rmi.server.hostname
with the desired value.
This is especially important when binding the RMI server host to the loopback
address for security reasons. See also
JMX Remote Access via SSH Tunnel.
Similarly to enabling JMX in a standalone Jetty server, you
enable the jmx-remote
module:
$ cd ${jetty.base} $ java -jar {$jetty.home}/start.jar --add-to-start=jmx-remote
When running Jetty embedded into an application, create and configure a ConnectorServer
:
Server server = new Server();
// Setup JMX
MBeanContainer mbeanContainer = new MBeanContainer(ManagementFactory.getPlatformMBeanServer());
server.addBean(mbeanContainer);
// Setup ConnectorServer
JMXServiceURL jmxURL = new JMXServiceURL("rmi", null, 1999, "/jndi/rmi:///jmxrmi");
ConnectorServer jmxServer = new ConnectorServer(jmxURL, "org.eclipse.jetty.jmx:name=rmiconnectorserver");
server.addBean(jmxServer);
The JMXServiceURL
above specifies that the RMI server binds to the wildcard address
on port 1999, while the RMI registry binds to the wildcard address on port 1099 (the
default RMI registry port).
The standard JMXConnectorServer
provides several options to authorize access.
For a complete guide to controlling authentication and authorization in JMX, see
Authentication and Authorization in JMX RMI connectors.
To authorize access to the JMXConnectorServer
you can use this configuration,
where the jmx.password
and jmx.access
files have the format specified in the blog entry above:
<New id="ConnectorServer" class="org.eclipse.jetty.jmx.ConnectorServer">
<Arg>
<New class="javax.management.remote.JMXServiceURL">
<Arg type="java.lang.String">rmi</Arg>
<Arg type="java.lang.String" />
<Arg type="java.lang.Integer">1099</Arg>
<Arg type="java.lang.String">/jndi/rmi:///jmxrmi</Arg>
</New>
</Arg>
<Arg>
<Map>
<Entry>
<Item>jmx.remote.x.access.file</Item>
<Item>
<New class="java.lang.String"><Arg><Property name="jetty.base" default="." />/resources/jmx.access</Arg></New>
</Item>
</Entry>
<Entry>
<Item>jmx.remote.x.password.file</Item>
<Item>
<New class="java.lang.String"><Arg><Property name="jetty.base" default="." />/resources/jmx.password</Arg></New>
</Item>
</Entry>
</Map>
</Arg>
<Arg>org.eclipse.jetty.jmx:name=rmiconnectorserver</Arg>
<Call name="start" />
</New>
Similarly, in code:
JMXServiceURL jmxURL = new JMXServiceURL("rmi", null, 1099, "/jndi/rmi:///jmxrmi");
Map<String, Object> env = new HashMap<>();
env.put("jmx.remote.x.access.file", "resources/jmx.access");
env.put("jmx.remote.x.password.file", "resources/jmx.password");
ConnectorServer jmxServer = new ConnectorServer(jmxURL, env, "org.eclipse.jetty.jmx:name=rmiconnectorserver");
jmxServer.start();
Calling ConnectorServer.start()
may be explicit as in the examples above,
or can be skipped when adding the ConnectorServer
as a bean to the Server
,
so that starting the Server
will also start the ConnectorServer
.
The JMX communication via RMI happens by default in clear-text.
It is possible to configure the ConnectorServer
with a SslContextFactory
so
that the JMX communication via RMI is encrypted:
<New id="ConnectorServer" class="org.eclipse.jetty.jmx.ConnectorServer">
<Arg>
<New class="javax.management.remote.JMXServiceURL">
<Arg type="java.lang.String">rmi</Arg>
<Arg type="java.lang.String" />
<Arg type="java.lang.Integer">1099</Arg>
<Arg type="java.lang.String">/jndi/rmi:///jmxrmi</Arg>
</New>
</Arg>
<Arg />
<Arg>org.eclipse.jetty.jmx:name=rmiconnectorserver</Arg>
<Arg><Ref refid="sslContextFactory" /></Arg>
</New>
Similarly, in code:
SslContextFactory sslContextFactory = new SslContextFactory();
sslContextFactory.setKeyStorePath();
sslContextFactory.setKeyStorePassword("secret");
JMXServiceURL jmxURL = new JMXServiceURL("rmi", null, 1099, "/jndi/rmi:///jmxrmi");
ConnectorServer jmxServer = new ConnectorServer(jmxURL, null, "org.eclipse.jetty.jmx:name=rmiconnectorserver", sslContextFactory);
It is possible to use the same SslContextFactory
used to configure the
Jetty ServerConnector
that supports TLS for the HTTP protocol.
This is used in the XML example above: the SslContextFactory
configured
for the TLS ServerConnector
is registered with an id of sslContextFactory
which is referenced in the XML via the Ref
element.
The keystore must contain a valid certificate signed by a Certification Authority.
The RMI mechanic is the usual one: the RMI client (typically a monitoring console) will connect first to the RMI registry (using TLS), download the RMI server stub that contains the address and port of the RMI server to connect to, then connect to the RMI server (using TLS).
This also mean that if the RMI registry and the RMI server are on different hosts, the RMI client must have available the cryptographic material to validate both hosts.
Having certificates signed by a Certification Authority simplifies by a lot the configuration needed to get the JMX communication over TLS working properly.
If that is not the case (for example the certificate is self-signed), then you need to specify the required system properties that allow RMI (especially when acting as an RMI client) to retrieve the cryptographic material necessary to establish the TLS connection.
For example, trying to connect using the JDK standard JMXConnector
with both
the RMI server and the RMI registry to domain.com
:
// System properties necessary for an RMI client to trust a self-signed certificate.
System.setProperty("javax.net.ssl.trustStore", "/path/to/trustStore");
System.setProperty("javax.net.ssl.trustStorePassword", "secret");
JMXServiceURL jmxURL = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://domain.com:1100/jmxrmi")
Map<String, Object> clientEnv = new HashMap<>();
// Required to connect to the RMI registry via TLS.
clientEnv.put(ConnectorServer.RMI_REGISTRY_CLIENT_SOCKET_FACTORY_ATTRIBUTE, new SslRMIClientSocketFactory());
try (JMXConnector client = JMXConnectorFactory.connect(jmxURL, clientEnv))
{
Set<ObjectName> names = client.getMBeanServerConnection().queryNames(null, null);
}
Similarly, to launch JMC:
$ jmc -vmargs -Djavax.net.ssl.trustStore=/path/to/trustStore -Djavax.net.ssl.trustStorePassword=secret
Note that these system properties are required when launching the ConnectorServer
too,
on the server, because it acts as an RMI client with respect to the RMI registry.
You can access JMX MBeans on a remote machine when the RMI ports are not open, for example because of firewall policies, but you have SSH access to the machine using local port forwarding via a SSH tunnel.
In this case you want to configure the ConnectorServer
with a JMXServiceURL
that binds the RMI server and the RMI registry to the loopback interface only:
service:jmx:rmi://localhost:1099/jndi/rmi://localhost:1099/jmxrmi
.
Then you setup the local port forwarding with the SSH tunnel:
$ ssh -L 1099:localhost:1099 <user>@<machine_host>
Now you can use JConsole or JMC to connect to localhost:1099
on your local
computer. The traffic will be forwarded to machine_host
and when there,
SSH will forward the traffic to localhost:1099
, which is exactly where
the ConnectorServer
listens.
When you configure ConnectorServer
in this way, you must set the system
property -Djava.rmi.server.hostname=localhost
, on the server.
This is required because when the RMI server is exported, its address and
port are stored in the RMI stub. You want the address in the RMI stub to be
localhost
so that when the RMI stub is downloaded to the remote client,
the RMI communication will go through the SSH tunnel.