OpenAM IPTables Rules

Firewalls are great. They always like getting in the way. As they should.

I had some problems setting up a particular instance of ForgeRock’s OpenAM product in that my firewall rules were blocking the product during the configuration stage. I hadn’t seen it previously because I’d built the product against environments with external firewalls.

When I attempted to configure my new OpenAM instance, I would get the following error:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Checking license acceptance...License terms accepted.
Checking configuration directory /usr/share/tomcat6/openam....Success.
Installing OpenAM configuration store...Success RSA/ECB/OAEPWithSHA1AndMGF1Padding.
Extracting OpenDJ, please wait...Complete
Running OpenDJ setupSetup command: --cli --adminConnectorPort 4444 --baseDN dc=openam,dc=com --rootUserDN cn=Directory Manager --ldapPort 50389 --skipPortCheck --rootUserPassword xxxxxxx --jmxPort 1689 --no-prompt --doNotStart --hostname openam002.openam.com
See /var/cache/tomcat6/temp/opendj-setup-1780262649001834132.log for a detailed log of this operation.

Configuring Directory Server ..... 
Done.

To see basic server configuration status and configuration you can launch /usr/share/tomcat6/openam/opends/bin/status

...Success.
...Success
Installing OpenAM configuration store in /usr/share/tomcat6/openam/opends...Success.
Creating OpenAM suffixYou have provided options for scheduling this operation as a task but optionsprovided for connecting to the server's tasks backend resulted in thefollowing error: 'Connect Error'

Error loading OpenAM suffix 1

emb.creatingfamsuffix.failure, refer to install.log under /usr/share/tomcat6/openam for more information.

The installation log referenced at the end there had the following contents:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
[root@OPENAM002 log]# cat /usr/share/tomcat6/openam/install.log
Checking license acceptance...License terms accepted.
License, legal-notices/license.txt, has been accepted.
License Hash: a5GIoWOZQaGncrUaRgtjo5kmm7g=.
Checking configuration directory /usr/share/tomcat6/openam....Success.
Installing OpenAM configuration store...Success RSA/ECB/OAEPWithSHA1AndMGF1Padding.
Extracting OpenDJ, please wait...Complete
Running OpenDJ setupSetup command: --cli --adminConnectorPort 4444 --baseDN dc=openam,dc=com --	rootUserDN cn=Directory Manager --ldapPort 50389 --skipPortCheck --rootUserPassword xxxxxxx --	jmxPort 1689 --no-prompt --doNotStart --hostname openam002.openam.com
See /var/cache/tomcat6/temp/opendj-setup-1780262649001834132.log for a detailed log of this operation.

Configuring Directory Server ..... Done.

To see basic server configuration status and configuration you can launch /usr/share/tomcat6/openam/opends/bin/status
...Success.
...Success
Installing OpenAM configuration store in /usr/share/tomcat6/openam/opends...Success.
Creating OpenAM suffixYou have provided options for scheduling this operation as a task but 	options
provided for connecting to the server's tasks backend resulted in the
following error: 'Connect Error'
Error loading OpenAM suffix 1
AMSetupServlet.processRequest: errorcom.sun.identity.setup.ConfiguratorException: \emb.creatingfamsuffix.failure
       at com.sun.identity.setup.EmbeddedOpenDS.setup(EmbeddedOpenDS.java:266)
       at com.sun.identity.setup.AMSetupServlet.setupEmbeddedDS(AMSetupServlet.java:964)
       at com.sun.identity.setup.AMSetupServlet.setupSMDatastore(AMSetupServlet.java:1020)
       at com.sun.identity.setup.AMSetupServlet.configure(AMSetupServlet.java:1092)
       at com.sun.identity.setup.AMSetupServlet.processRequest(AMSetupServlet.java:693)
       at com.sun.identity.config.wizard.Wizard.createConfig(Wizard.java:304)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:606)
       at org.apache.click.util.ClickUtils.invokeMethod(ClickUtils.java:3317)
       at org.apache.click.util.ClickUtils.invokeListener(ClickUtils.java:2088)
       at org.apache.click.control.AbstractControl$1.onAction(AbstractControl.java:228)
       at org.apache.click.ActionEventDispatcher.fireActionEvent(ActionEventDispatcher.java:259)
       at org.apache.click.ActionEventDispatcher.fireActionEvents(ActionEventDispatcher.java:236)
       at org.apache.click.ActionEventDispatcher.fireActionEvents(ActionEventDispatcher.java:180)
       at org.apache.click.ClickServlet.performOnProcess(ClickServlet.java:746)
       at org.apache.click.ClickServlet.processAjaxPageEvents(ClickServlet.java:1860)
       at org.apache.click.ClickServlet.processPage(ClickServlet.java:559)
       at org.apache.click.ClickServlet.handleRequest(ClickServlet.java:383)
       at org.apache.click.ClickServlet.doGet(ClickServlet.java:276)
       at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
       at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
       at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
       at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
       at org.forgerock.openam.validation.ResponseValidationFilter.doFilter(ResponseValidationFilter.java:44)
       at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
       at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
       at com.sun.identity.setup.AMSetupFilter.doFilter(AMSetupFilter.java:113)
       at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
       at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
       at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
       at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
       at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
       at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
       at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
       at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
       at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
       at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
       at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
       at java.lang.Thread.run(Thread.java:745)

The cause of this problem turned out to be just local traffic being blocked (in particular, port 4444). The fastest way to determine that was the problem was to simply run:

1
2
service iptables stop
chkconfig iptables off

And then run the configuration again, which would succeed. The part I forgot in my iptables script was:

1
iptables -A OUTPUT -o lo -j ACCEPT

Here’s the full iptables script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
#!/bin/bash

# Flush Existing Rules
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X

# Set Default Policy to Drop Packets
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT DROP

# Allow Localhost Traffic, for internal services
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

# Allow inbound packs part of existing connections
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

# Allow DNS (dport needs to be specified for output, as sport will be random)
iptables -A INPUT -p tcp -m tcp --dport 53 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -p udp -m udp --dport 53 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m tcp --dport 53 -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p udp -m udp --dport 53 -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m tcp --sport 53 -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p udp -m udp --sport 53 -m state --state NEW,RELATED,ESTABLISHED -j ACCEPT

# Allow SSH, HTTP and HTTPS
iptables -A INPUT -p tcp -m multiport --dports 22,80,443 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m multiport --dports 22,80,443 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m multiport --sports 22,80,443 -m state --state ESTABLISHED -j ACCEPT

# Allow LDAP (both incoming and outgoing requests)
iptables -A INPUT -p tcp -m multiport --dports 389,636 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m multiport --dports 389,636 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m multiport --sports 389,636 -m state --state ESTABLISHED -j ACCEPT

# Allow Tomcat Services
iptables -A INPUT -p tcp -m multiport --dports 8080,8081,8443 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m multiport --dports 8080,8081,8443 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m multiport --sports 8080,8081,8443 -m state --state ESTABLISHED -j ACCEPT

# Allow access to OpenAM Services
iptables -A INPUT -p tcp -m multiport --dports 1689,2689,4444,5444,50389,51389,50899,58989 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m multiport --dports 1689,2689,4444,5444,50389,51389,50899,58989 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -m multiport --sports 1689,2689,4444,5444,50389,51389,50899,58989 -m state --state ESTABLISHED -j ACCEPT

# Allow inbound and outbound pings
iptables -A INPUT -p icmp -j ACCEPT
iptables -A OUTPUT -p icmp -j ACCEPT

# Log before dropping
iptables -A INPUT -j LOG  -m limit --limit 12/min --log-level 4 --log-prefix 'IP INPUT drop: '
iptables -A INPUT -j DROP
	
iptables -A OUTPUT -j LOG  -m limit --limit 12/min --log-level 4 --log-prefix 'IP OUTPUT drop: '
iptables -A OUTPUT -j DROP

exit 0