WEBVTT NOTE This file was generated by Descript 00:00:00.316 --> 00:00:03.796 Charles Frye: This video walks through the instructions for setting up the compute 00:00:03.796 --> 00:00:08.296 environment for the FSDL 2022 labs. 00:00:09.736 --> 00:00:12.916 These instructions assume that you have a Linux machine with a GPU 00:00:12.921 --> 00:00:15.196 available that you can SSH into. 00:00:15.496 --> 00:00:18.866 If that's not you, then we recommend that you stick with Colab. 00:00:19.006 --> 00:00:21.886 So check out that video for your instructions. 00:00:22.240 --> 00:00:23.140 Couple of side notes. 00:00:23.140 --> 00:00:26.470 If you have a Windows machine with a GPU and you have Windows 00:00:26.470 --> 00:00:30.010 Subsystem Linux 2, you're welcome to try out these instructions, but 00:00:30.010 --> 00:00:31.271 we can't guarantee they'll work. 00:00:31.271 --> 00:00:34.251 Cuz Windows Subsystem Linux with GPU support is a very fresh 00:00:34.256 --> 00:00:35.753 technology with sharp edges. 00:00:35.753 --> 00:00:36.983 You can read more about it there. 00:00:37.033 --> 00:00:40.377 If you have a Mac with an Apple Silicon GPU, don't even try to 00:00:40.382 --> 00:00:41.997 install a deep learning stack on it. 00:00:41.997 --> 00:00:46.184 A lot of good engineers have spent a lot of hours Trying to get 00:00:46.184 --> 00:00:49.695 libraries to compile and run on that hardware, just run things on Colab. 00:00:50.146 --> 00:00:55.032 So the instructions for local development can be found on GitHub here. 00:00:55.901 --> 00:00:59.711 in order to use them, the first step is to check that repo out. 00:00:59.739 --> 00:01:02.066 So I'm doing this on a fresh Ubuntu machine. 00:01:03.865 --> 00:01:07.737 And once we've cloned the repo, we can go inside, take a look at it. 00:01:08.854 --> 00:01:13.864 The next step is to set up the Python environment using Anaconda, 00:01:13.984 --> 00:01:17.224 which is our system package and Python environment manager. 00:01:17.881 --> 00:01:21.241 So to install it, you can follow the instructions at that link to install 00:01:21.241 --> 00:01:23.381 a Python 3.7 version of miniconda. 00:01:23.431 --> 00:01:25.185 If you install another version, it's okay. 00:01:25.185 --> 00:01:27.539 You'll just have multiple versions of Python on your machine. 00:01:27.926 --> 00:01:30.446 If you only have terminal access to your machine, then you'll need 00:01:30.446 --> 00:01:34.176 to use something like wget to get the installer for miniconda. 00:01:34.196 --> 00:01:36.866 Otherwise the instructions will direct you how to do it via 00:01:36.871 --> 00:01:38.306 your graphical user interface. 00:01:39.486 --> 00:01:41.696 You can run the installer from the command line. 00:01:41.872 --> 00:01:45.160 You just need to type bash and then the name of the installer 00:01:45.160 --> 00:01:46.480 file that you downloaded. 00:01:48.423 --> 00:01:50.223 You can accept the default options 00:01:53.725 --> 00:01:56.745 Here though we wanna say yes and use conda init 00:01:59.228 --> 00:02:02.108 at this point, you'll likely need to close and reopen your terminal. 00:02:03.240 --> 00:02:06.420 Notice that my prompt now includes the name of the Anaconda environment. 00:02:06.450 --> 00:02:08.310 This is the default behavior of Anaconda. 00:02:08.932 --> 00:02:10.252 Back inside the lab repo. 00:02:10.282 --> 00:02:13.852 There's a make file that automates pretty much the entirety of the rest of setup. 00:02:13.912 --> 00:02:17.136 So to use it, you need the make command, and then the make target 00:02:17.176 --> 00:02:19.416 conda update from that file. 00:02:38.560 --> 00:02:39.500 That takes some time. 00:02:39.564 --> 00:02:42.654 but now we've got our system libraries like cuda and Python. 00:02:43.134 --> 00:02:46.314 And all the things that cuda and cudnn rely on. 00:02:46.539 --> 00:02:50.849 The last thing we need to do is run that command at the bottom there. 00:02:50.899 --> 00:02:55.125 conda activate fsdl-text-recognizer-2022. 00:02:56.797 --> 00:02:59.107 So that sets up our system libraries, but it doesn't 00:02:59.107 --> 00:03:00.547 install everything that we need. 00:03:00.607 --> 00:03:05.227 Everything else gets installed using the Python package manager, pip. 00:03:05.306 --> 00:03:07.404 but we provide a make target for that as well. 00:03:07.494 --> 00:03:10.018 So all you need to do is type, make pip-tools. 00:03:25.974 --> 00:03:26.184 All right. 00:03:26.184 --> 00:03:27.743 We've got the environment installed. 00:03:27.743 --> 00:03:30.823 The last step that we need to do is set the Python path, which makes 00:03:30.823 --> 00:03:33.728 sure that we're able to import all the libraries in the class. 00:03:33.999 --> 00:03:35.229 the command to do it is simple. 00:03:35.229 --> 00:03:38.329 It just sets an environment variable of the $PYTHONPATH. 00:03:38.349 --> 00:03:40.550 That includes the current directory, which is 00:03:43.329 --> 00:03:46.209 ".". And now we should be able to enter into one of the lab directories and 00:03:46.209 --> 00:03:47.769 import the text_recognizer code. 00:03:49.293 --> 00:03:52.273 The next time that you log into this same machine, the conda 00:03:52.293 --> 00:03:56.100 environment will be back to base and the PYTHONPATH won't be set. 00:03:56.100 --> 00:03:58.320 you'll have to activate the conda environment every time 00:03:58.325 --> 00:03:59.818 with that conda activate command. 00:04:01.200 --> 00:04:05.030 But for the PYTHONPATH, you could just edit your shell configuration file. 00:04:05.030 --> 00:04:06.840 So for most folks that'll be .bashrc. 00:04:08.926 --> 00:04:11.910 If you have your machine locally, and it's got, a graphical user 00:04:11.910 --> 00:04:14.760 interface and a browser, then you're done and ready to go, and you can 00:04:14.760 --> 00:04:18.059 just type in "jupyter notebook" and you should get a browser window 00:04:18.064 --> 00:04:19.889 with a Jupyter server running in it. 00:04:20.183 --> 00:04:23.153 Once you reach this screen, you're ready to go to try out the labs. 00:04:23.153 --> 00:04:26.348 for the first lab, with the overview of our text recognizer 00:04:26.353 --> 00:04:30.538 architecture, we can just click overview.ipynb and we're ready to go. 00:04:31.230 --> 00:04:35.700 But if you are SSHing into a machine, maybe it's a cloud instance. 00:04:35.723 --> 00:04:38.328 then there's a little bit more setup that will go through. 00:04:38.911 --> 00:04:41.881 So if we type in jupyter notebook here, 00:04:42.809 --> 00:04:45.726 we'll see that the server's running, but if you're SSH into 00:04:45.726 --> 00:04:51.039 a machine, then you can't use the local host URL that's on there. 00:04:51.066 --> 00:04:53.916 cuz that will point to whatever machine you're SSHing from. 00:04:53.939 --> 00:04:57.449 there are instructions on how to set up a Jupyter server for a 00:04:57.449 --> 00:04:59.635 single user that you can check out. 00:04:59.635 --> 00:05:02.923 but there's a slightly simpler way to do it that we'll walk through now. 00:05:03.096 --> 00:05:05.547 So you'll need another session on the same machine. 00:05:05.597 --> 00:05:09.210 If you're familiar with something like tmux, you can do that or 00:05:09.210 --> 00:05:10.680 just open up another terminal 00:05:10.810 --> 00:05:14.830 and then we're gonna use a tool called ngrok to connect to our jupyter server. 00:05:15.140 --> 00:05:17.540 If you don't have ngrok, type the name and then follow the 00:05:17.540 --> 00:05:20.750 instructions for how to install using your system's package manager. 00:05:22.686 --> 00:05:25.536 Then in the URL for the Jupyter notebook, you'll see a port 00:05:25.536 --> 00:05:27.744 number after the name localhost. 00:05:28.367 --> 00:05:31.430 So type that port number after ngrok http. 00:05:31.507 --> 00:05:33.277 So for me, that's 8888. 00:05:33.427 --> 00:05:34.147 That's the default. 00:05:35.199 --> 00:05:39.139 And now we have a connection to that Jupyter server being run by ngrok. 00:05:39.322 --> 00:05:42.262 So you can see the URL here where the forwarding is happening. 00:05:42.262 --> 00:05:44.596 Let's copy that and open it in a browser window. 00:05:46.449 --> 00:05:49.359 So when you first try this, you'll get an error that says that you haven't 00:05:49.359 --> 00:05:51.249 signed up for an ngrok account. 00:05:51.249 --> 00:05:52.299 So go ahead and do that. 00:05:52.384 --> 00:05:56.254 And get a hold of an auth token and follow the instructions for adding 00:05:56.404 --> 00:05:58.534 your authentication token here. 00:05:59.737 --> 00:06:01.537 So I've added my authentication token. 00:06:01.786 --> 00:06:03.856 And now I can run that same ngrok command, 00:06:05.406 --> 00:06:06.546 copy the URL. 00:06:08.885 --> 00:06:10.985 And if I open that URL in a browser window, now I'm 00:06:10.985 --> 00:06:12.755 connected to my Jupyter server. 00:06:13.249 --> 00:06:16.596 the first time you log in, it should ask you for a token. 00:06:16.626 --> 00:06:19.086 That's also in the output of the Jupyter server. 00:06:19.206 --> 00:06:20.106 So let's take a look. 00:06:20.730 --> 00:06:21.450 Here's my token. 00:06:21.644 --> 00:06:25.304 You can switch it to a password by entering into the field down here. 00:06:25.530 --> 00:06:28.000 If you run into trouble, just make sure that you copied that token 00:06:28.000 --> 00:06:31.267 correctly without any spaces or any of the other stuff on the line. 00:06:31.267 --> 00:06:33.489 Just everything after the equal sign.