Entries tagged [distributed]
Starting Apache Falcon in distributed mode
Apache Falcon can be deployed in two modes, Embedded mode and Distributed mode. Both the modes have there uses which are described below : 1. Embedded Mode : Embedded mode is also called standalone mode. This setup is useful when you have only one cluster from where you want data to be managed and processed. All the request like submit, delete, schedule etc go to the single server that is started.
To start falcon in distributed mode the following steps are required :1. Build falcon in distributed mode 2. Edit startup property for falcon server / servers 3. Edit runtime property for falcon prism 4. Example
Step 1. Build falcon in distributed modePlease follow the steps here for building falcon in distributed mode. If above step is successful you should have 3 ".tar.gz" files at the end ,one each for falcon prism, server and client in the target dirs.
Step 2. Edit startup property for falcon serverFor starting falcon in distributed we need to provide logical names to servers we are starting. These names need to be mentioned in startup.properties for the given server. Add the following property in the "conf/startup.properties" in the server package you have untar.
*.current.colo=< colo name >Colo is a colocation or a physical data centre location where one or more clusters may be configured. If you are planning to start more than one server add the above in each of the conf/startup.properties . Remember the cole name you have mentioned here. Same name should be used in cluster.xml while submitted a cluster later via prism. Start the servers using the commands given here here
Step 3. Edit runtime property for falcon prismUntar the falcon prism package. Once servers have been configured and started, prism need to be configured with information on where these servers are started. For that below two properties needs to be added to prism "conf/runtime.properties" .
*.all.colos=< colo name 1 >, < colo name 2 > *.falcon.< colo name 1 >.endpoint=http://< hostname 1 >:15000 *.falcon.< colo name 2 >.endpoint=http://< hostname 2 >:15000Start Falcon Prism using commands given here here
Example :If you are configuring a falcon setup with 1 prism and 2 servers, with cluster name cluster1 and cluster2, below are the thing that needs to added : 1. edit server one startup.properties and add *.current.colo=colo1 2. edit server two startup.properties and add *.current.colo=colo2 3. Start the servers 4. edit prism runtime.properties and add
*.all.colos=colo1, colo2 *.falcon.colo1.endpoint=http://< hostname 1 >:15000 *.falcon.colo2.endpoint=http://< hostname 2 >:150005. Start falcon prism.
Note:By default falcon server start with its own activemq, if for any reason you dont want embedded activemq (like starting more than one server on same host) , it can be disabled. Step 1. Disable start of activemq in falcon server start script edit "bin/falcon-start" script and add FALCON_OPTS="-Dfalcon.embeddedmq=false" Step 2. In startup.properties of falcon server add your own activemq url