Model 2: running JupyterLab on a remote server where your datasets live
Maybe you’ve heard the saying that “data science is anything that causes Excel to crash” The next stage after that is anything that doesn’t fit on a laptop. As your experiments scale, you may run into either
- not enough compute on your laptop
- not enough storage on your laptop
Either way, the solution will probably be for you to get a beefy dev server from your DevOps/IT team. That dev server will be hosted in a lab somewhere that you never physically see. You just access it remotely.
Or maybe it’s simply WFH during Covid that’s caused you to work remotely with machines that you usually work directly on.
The challenge with starting Jupyter on a remote machine is accessing the UI.
Scenario A: there’s no firewall between you and the remote machine
Add the --ip
parameter to your jupyter launch command. It sets the IP address that the notebook server listens on (default value = “localhost”).
jupyter notebook --ip=<remote server IP address> --port=8888
In your laptop browser, instead of navigating to http://localhost:8888/, you can connect directly to the server at http://<remote server IP address>:8888/ instead.
Get fancy:
Are you restarting your Jupyter on your remotehost every time you open your laptop?You can use a terminal multiplexer such as tmux or screen to essentially detach the session on your remotehost. That way, the session doesn’t depend on the connection from your localhost.
For a comparison between tmux and screen, you can check out this StackExchange post , but as the header warning states, it’s an opinion-based question.
Scenario B: your notebook server is running somewhere behind a firewall, like in the public cloud.
Instructions vary per cloud vendor (see below), but the high level steps are:
- Launch jupyter from within your remote server
- From your local machine, configure port forwarding.
Port forwarding?
When you start a Jupyter server, by default, the GUI is hosted on a port on the machine where the server is launched . In this scenario, port forwarding tells your local machine to take any traffic for a specific local port and forward it to a port on a remote machine. ( Further reading: The Missing Semester of Your CS Education )Vendor specific instructions:
AWS
– Official documentation: Set up Jupyter for the AWS Deep Learning AMI
GCP
– Official documentation: Connecting to JupyterLab for the Deep Learning VM Image
– User example (includes changing your instance’s firewall settings as an alternative to port forwarding)
Azure
– Official documentation: Configure Jupyter in a Azure Machine Learning dev environment (uses Azure Notebooks as the server)
Related Articles
Request for deletion
About
MC.AI – Aggregated news about artificial intelligence
MC.AI collects interesting articles and news about artificial intelligence and related areas. The contributions come from various open sources and are presented here in a collected form.
The copyrights are held by the original authors, the source is indicated with each contribution.
Contributions which should be deleted from this platform can be reported using the appropriate form (within the contribution).
MC.AI is open for direct submissions, we look forward to your contribution!
Search on MC.AI
mc.ai aggregates articles from different sources - copyright remains at original authors
以上就是本文的全部内容，希望对大家的学习有所帮助，也希望大家多多支持 码农网
猜你喜欢:
本站部分资源来源于网络，本站转载出于传递更多信息之目的，版权归原作者或者来源机构所有，如转载稿涉及版权问题，请联系我们。
Data-intensive Text Processing With Mapreduce
Jimmy Lin、Chris Dyer / Morgan and Claypool Publishers / 2010-4-30 / USD 40.00
Our world is being revolutionized by data-driven methods: access to large amounts of data has generated new insights and opened exciting new opportunities in commerce, science, and computing applicati......一起来看看 《Data-intensive Text Processing With Mapreduce》 这本书的介绍吧!
