Google Colab, a serverless notebook service provided by Google, offers the hosted runtime (kernel) with 3 options:
1) without GPU: 2 CPUs. 13 GB RAM, 50 GB disk
2) with GPU: 2 CPUs, 13 GB RAM, 358 GB disk
3) with TPU: 2 CPUs, 13 GB, 50 GB disk
In most cases, the offered resource (CPU, RAM, and disk) is sufficient, but sometimes we need more resources, e.g., more CPUs to speed up the process, more RAMs to process the data, and more disks to store the data.
Fortunately, Google Colab provides its users with local runtime that could be ported to its service. Here are the steps to port local runtime:
1) We need to install Jupyter’s forwarding service
pip install jupyter_http_over_ws jupyter serverextension enable --py jupyter_http_over_ws
2) Run the Jupyter service
python -m jupyter notebook --ip=0.0.0.0 --port=58888 --no-browser --NotebookApp.token='' --NotebookApp.password='' --NotebookApp.allow_origin='https://colab.research.google.com' --NotebookApp.port_retries=0
3) [optional] Since I’m using a remote machine (high-performance cluster (HPC) TU Delft), I need to ssh forward the relevant port to my local computer.
ssh -nNT -L 0.0.0.0:58888:[HPC_IP]:58888 [username]@[transit_machine_ip]
4) Open Google Colab: https://colab.research.google.com; Create a new notebook; select Connect to local runtime; input 58888 as the port
Voila! Google Colab now runs on local runtime.