How to connect DAS instance with an external spark cluster

Suppose you have a DAS instance in one node and External spark cluster in another node.

Now, you need to connect DAS instance with the external spark cluster.

First, unzip another DAS instance in node, where the external spark cluster has located.

Then, create a symlink(symbolic link) to DAS_HOME as below in /mnt/das location (you can choose any location you want) of your node.

cd    /mnt/das/das_symlink

cd   /mnt/das/

remove current symlink and create another one by specifying your DAS instance home location as below.

sudo ln -s  /home/centos/das/wso2das-3.0.0-SNAPSHOT das_symlink

Then it will create a symlink in /mnt/das location.

Give this symlink location to DAS_HOME/repository/conf/analytics/spark/spark_default.config file as below.

carbon.das.symbolic.link  /mnt/das/das_symlink

Then go to external spark cluster located node and start the spark cluster.

Create the symlink  by following above procedure for the DAS instance which is going to connect with external spark but not in external spark cluster node.

note that both DAS instances in external spark cluster and other node should have to have exactly the same configurations like databases etc…

(Also don’t start the DAS instance in external spark cluster node.)

Now, if you go to external-spark-node-IP:4040 you will see the following web page.

The spark.driver.extraClassPath should have this symlink location.(mnt/das/das_symlink)

Then, start the DAS node to connect with external spark cluster.

All the server instances should work properly, if the configurations has set properly.