概述
深度学习数据集制作工作 Creating my workstation has been a dream for me, if nothing else. 创建工作站对我来说是一个梦想,即使没有其他选择。 I knew the process involved, yet I somehow never got to it. It might have been time or money. Mostly Money. 我知道所涉及的过程,但是我莫名其妙地从不进行。 可能是时间或金钱。 主要是金钱。 But this time I just had to do it. I was just fed up with setting up a server on AWS for any small personal project and fiddling with all the installations. Or I had to work on Google Collab notebooks, which have a lot of limitations on running times and network connections. So, I found out some time to create a Deep Learning Rig with some assistance from NVIDIA folks. 但是这次我只需要这样做。 我刚受够在AWS上为任何小型个人项目设置服务器并摆弄所有安装。 或者,我必须使用Google Collab笔记本,因为笔记本在运行时间和网络连接方面有很多限制。 因此,我花了一些时间在NVIDIA员工的帮助下创建深度学习装备。 The whole process involved a lot of reading up and watching a lot of Youtube videos from Linus Tech Tips. And as it was the first time I was assembling a computer from scratch, it was sort of special too. 整个过程涉及大量阅读和观看Linus Tech Tips的许多Youtube视频。 这是我第一次从头开始组装计算机,这也很特别。 Building the DL rig as per your requirements takes up a lot of research. I researched on individual parts, their performance, reviews, and even the aesthetics. 根据您的要求构建DL装备需要大量的研究。 我研究了各个部分,它们的性能,评论,甚至美学。 Now, most of the workstation builds I researched were focussed on gaming, so I thought of putting down a Deep Learning Rig Spec as well. 现在,我研究的大多数工作站版本都集中在游戏上,因此我也考虑制定《深度学习装备规范》。 I will try to put all the components I used along with the reasons why I went with those particular parts as well. 我将尝试列出我使用的所有组件以及为什么要使用这些特定部分。 If building your own seems too difficult or expensive, Exxact has a great line of deep learning workstations and servers starting at $5,899, with a couple of NVIDIA RTX 2080 Ti GPUs, Intel Core i9, 3-year warranty, and a deep learning software stack. 如果构建自己的产品似乎太困难或太昂贵,Exxact拥有大量的深度学习工作站和服务器,起价为$ 5,899,配备了两个NVIDIA RTX 2080 Ti GPU,Intel Core i9、3年保修和深度学习软件堆栈。 Also, if you want to see how I set up the Deep Learning libraries after setting up the system to use Ubuntu 18.04, you can view this definitive guide for Setting up a Deep Learning Workstation. 此外,如果要在设置系统以使用Ubuntu 18.04之后查看我如何设置深度学习库,则可以查看此权威指南,以设置深度学习工作站 。 The very first answer that comes to my mind is, why not? 我想到的第一个答案是,为什么不呢? I work a lot on deep learning and machine learning applications, and it always has been such a massive headache to churn up a new server and installing all the dependencies every time I start to work on a new project. 我在深度学习和机器学习应用程序上做了大量工作,每次我开始从事新项目时,总是要组装一个新服务器并安装所有依赖项,这一直是一个巨大的难题。 Also, it looks great, sits on your desk, is available all the time, and is open to significant customization as per your requirements. 此外,它看起来很棒,可以坐在您的办公桌上,随时可用,并且可以根据您的要求进行重大定制。 Adding to this the financial aspects of using the GCP or AWS, and I was pretty much sold on the idea of building my rig. 再加上使用GCP或AWS的财务方面,就建造钻机的想法我几乎被甩了。 It took me a couple of weeks to come up with the final build. 我花了几个星期才得出最终版本。 I knew from the start that I want to have a lot of computing power and also something that would be upgradable in the coming years. Currently, my main priorities were to get a system that could support two NVIDIA RTX Titan cards with NVLink. That would allow me to have 48GB GPU memory at my disposal. Simply awesome. 从一开始,我就知道我希望拥有强大的计算能力,并且希望在未来几年内能够对其进行升级。 目前,我的主要任务是获得一个系统,该系统可以支持两块带有NVLink的NVIDIA RTX Titan卡。 这样一来,我就可以拥有48GB的GPU内存。 就是棒。 PS: The below build might not be the best build, and there may be cheaper alternatives present, but I know for sure that it is the build with the minimal future headache. So I went with it. I also contacted Nvidia to get a lot of suggestions about this particular build and only went forward after they approved of it. PS: 以下版本可能不是最佳版本,并且可能存在更便宜的替代方案,但我可以肯定,这是将来头痛最小的版本。 所以我同意了。 我还与Nvidia联系,以获取有关此特定版本的很多建议,直到他们批准后才继续。 Yes, I went with an Intel processor and not an AMD one. My reason for this (though people may differ with me on this) is because Intel has more compatible and related software like Intel’s MKL, which benefits most of the Python libraries I use. 是的,我选择了Intel处理器而不是AMD处理器。 我这样做的原因(尽管人们对此可能有所不同)是因为英特尔拥有更多兼容和相关的软件,例如英特尔的MKL,它使我使用的大多数Python库受益。 Another and maybe a more important reason, at least for me, was that it was suggested by the people at NVIDIA to go for i9 if I wanted to have a dual RTX Titan configuration. Again zero headaches in the future. 至少对我来说,另一个也许是更重要的原因是,如果我想拥有双RTX Titan配置,NVIDIA的人建议选择i9。 将来再次零头痛。 So why this particular one from the Intel range? 那么,为什么要选择这种来自英特尔的产品呢? I started with 9820X with its ten cores and 9980XE with 18 cores, but the latter stretched my budget a lot. I found that i9–9920X, with its 12 cores and 3.5 GHz processor, fit my budget just fine, and as it is always better to go for the mid-range solution, I went with it. 我开始与9820X具有其10核心和9980XE 18个内核,但后者伸出我的预算了不少。 我找到 i9-9920X ,其12个内核和3.5 GHz处理器,适合我的预算就好了,和它始终是更好地去为中档的解决方案,我用它去。 Now a CPU is the component that decides a lot of other components you are going to end up using. 现在,CPU是决定您将要使用的许多其他组件的组件。 For example, if you choose an i9 9900X range of CPU, you will have to select an X299 motherboard, or if you are going to use an AMD Threadripper CPU, you will need an X399 Motherboard. So be mindful of choosing the right CPU and motherboard. 例如,如果您选择i9 9900X系列的CPU,则必须选择X299主板,或者如果您要使用AMD Threadripper CPU ,则需要X399主板。 因此,请注意选择正确的CPU和主板。 This was a particularly difficult choice. There are just too many options here. I wanted a Motherboard that could support at least 96GB RAM (again as per the specifications by the NVIDIA Folks for supporting 2 TITAN RTX GPUs). That meant that I had to have at least six slots if I were to use 16GB RAM Modules as 16x6=96. I got 8 in this one, so it is expandable till 128 GB RAM. 这是一个特别困难的选择。 这里有太多选择。 我想要一个能够支持至少96GB RAM的主板(同样按照NVIDIA Folks的规范,支持2个TITAN RTX GPU)。 这意味着如果要使用16GB RAM模块作为16x6 = 96,则必须至少有六个插槽。 我有8个,因此它可以扩展到128 GB RAM。 I also wanted to be able to have 2 TB NVMe SSD in my system(in the future), and that meant I needed 2 M.2 ports, which this board has. Or else I would have to go for a much expensive 2TB Single NVMe SSD. 我还希望能够在我的系统中拥有2 TB NVMe SSD(将来),这意味着我需要2个M.2端口,该主板具有此端口。 否则,我将不得不购买价格昂贵的2TB Single NVMe SSD。 I looked into a lot of options, and based on the ATX Form factor, 4 PCI-E x16 slots, and the reasonable pricing of the board, I ended up choosing this one. 我看着很多的选择,以及基于ATX板型,4条PCI-E x16插槽,和董事会的合理定价,我最终选择了这一个 。 Liquid cooling is in rage right now. And initially, I also wanted to go for an AIO cooler, i.e., liquid cooling. 液体冷却现在很流行。 最初,我还想购买AIO冷却器,即液体冷却器。 But after talking to a couple of people at NVIDIA as well as scrouging through the internet forums on the pro and cons of both options, I realized that Air cooling is better suited to my needs. So I went for the Noctua NH-D15, which is one of the best Air coolers in the market. So, I went with the best air cooling instead of a mediocre water cooling. And this cooler is SILENT. More on this later. 但是,在与NVIDIA的几个人进行了交谈,并在互联网论坛上讨论了这两种方案的利弊之后,我意识到风冷技术更适合我的需求。 因此,我购买了Noctua NH-D15 ,它是市场上最好的空气冷却器之一。 因此,我选择了最佳的空气冷却系统,而不是普通的水冷系统。 而且这个散热器很安静。 稍后再详细介绍。 The next thing to think was a case that is going to be big enough to handle all these components and also be able to provide the required cooling. It was where I spent most of my time while researching. 接下来要考虑的情况是,这种情况的大小将足以容纳所有这些组件,并能够提供所需的冷却。 在这里,我大部分时间都在研究上。 I mean, we are going to keep 2 TITAN RTX, 9920x CPU, 128 GB RAM. It’s going to be a hellish lot of heat in there. 我的意思是,我们将保留2个TITAN RTX,9920x CPU和128 GB RAM。 那里将会有很多地狱般的热量。 Add to that the space requirements for the Noctua air cooler and the capability to add a lot of fans, and I was left with two options based on my poor aesthetic sense as well as the availability in my country. The options were — Corsair Air 540 ATX and the Phanteks Enthoo Pro Tempered Glass PH-ES614PTG_SWT. 再加上Noctua空气冷却器的空间要求以及增加很多风扇的能力,基于我的审美观念不佳以及我所在的国家的可用性,我剩下了两个选择。 选项为— Corsair Air 540 ATX 和Phanteks Enthoo Pro钢化玻璃PH-ES614PTG_SWT 。 Both of them are exceptional cases, but I went through with the Enthoo Pro as it is a more recently launched case and has a bigger form factor(Full Tower) offers options for more customizable build in the future too. 两者都是例外情况,但是我使用了Enthoo Pro,因为这是一个较新推出的案例,并且具有更大的尺寸(“ Full Tower”)也为将来提供更多可自定义的版本提供了选择。 These 2 TITAN RTX are by far the most important and expensive part of the whole build. These alone take up 80% of the cost, but aren’t they awesome? 这两个TITAN RTX 到目前为止,它是整个构建中最重要和最昂贵的部分。 仅这些一项就占了成本的80%,但是它们不是很棒吗? I wanted to have a high-performance GPU in my build, and the good folks at NVIDIA were generous enough to send me two of these to test out. 我想在自己的构建中使用高性能的GPU,而NVIDIA的好伙伴足够慷慨地向我发送其中两个进行测试。 I just love them. The design. The way they look in the build and the fact that they can be combined using a 3 Slot NVLink to provide 48 GB of GPU RAM effectively. Just awesome. If money is an issue, 2 x NVIDIA GeForce RTX 2080 Ti would also work fine as well. Only a problem will be that you might need smaller batch sizes training on RTX 2080 Ti, and in some cases, you might not be able to train large models as RTX2080Ti has 11GB RAM only. Also, you won’t be able to use NVLink, which combines the VRAM of multiple GPUs in TITANs. 我就是爱他们 该设计。 它们在构建中的外观以及可以使用3插槽NVLink进行组合以有效提供48 GB GPU RAM的事实。 太棒了 如果有钱问题,那么2 x NVIDIA GeForce RTX 2080 Ti也可以正常工作。 唯一的问题是,您可能需要在RTX 2080 Ti上进行较小批量的培训,并且在某些情况下,由于RTX2080Ti仅具有11GB RAM,因此您可能无法培训大型模型。 而且,您将无法使用NVLink,它会在TITAN中组合多个GPU的VRAM。 What about storage? NVMe SSD, of course, and the Samsung Evo Plus is the unanimous and most popular winner in this SSD race. 那存储呢? 当然,NVMe SSD和Samsung Evo Plus在这场SSD竞赛中是一致且最受欢迎的赢家。 I bought 1 of them till now, but as I have 2 M.2 ports in my motherboard, I will get total storage of 2TB SSD in the future. 到目前为止,我已经购买了其中的1个,但是由于我的主板上有2个M.2端口,将来我将获得2TB SSD的总存储量。 You can also get a couple of 2.5" SSD for more storage space. 您还可以购买几个2.5英寸SSD,以获取更多存储空间。 I wanted to have a minimum of 96GB RAM, as suggested by the NVIDIA team. So I said what the heck and went with the full 128 GB RAM without cheaping out. 根据NVIDIA团队的建议,我希望至少有96GB的RAM。 因此,我说了什么,不带便宜就配备了完整的128 GB RAM。 As you can see, these RAM sticks are not RGB lit, and that is a conscious decision as the Noctua Air Cooler doesn’t provide a lot of clearance for RAM Slots and the RGB ones had a slightly higher height. So keep that in mind. Also, I was never trying to go for an RGB Build anyway as I want to focus on those lit up TITANs in my build. 如您所见,这些RAM棒没有RGB点亮,这是一个有意识的决定,因为Noctua空气冷却器不能为RAM插槽提供很多间隙,而RGB插槽的高度略高。 所以记住这一点。 另外,我从来没有尝试过进行RGB构建,因为我想专注于构建中那些照亮的TITAN。 A 1200W power supply is a pretty big one, but that is needed realizing that the estimated wattage of our components at full wattage is going to be ~965W. 1200W电源是一个很大的电源,但要意识到我们的组件在全瓦时的估计瓦数将约为965W,这是必需的。 I had a couple of options for the power supply from other manufacturers also but went with this one because of Corsair’s name. I would have gone with HX1200i, but it was not available, and AX1200i was much more expensive than this one at my location. But both of them are excellent options apart from this one. 我也有其他制造商提供的几种电源选择,但由于海盗船的名字,我选择了这种电源。 我本来会选择HX1200i的 ,但它不可用,而AX1200i在我的位置比这本要贵得多。 但是,除了这一项以外,它们都是极好的选择。 The Phanteks case comes up with three fans, but I was recommended to upgrade the intake, and exhaust fans of the case to BeQuiet BL071 PWM Fans as Dual Titans can put out a lot of heat. I have noticed that the temperature of my room is almost 2–3 degrees higher than the outside temperature, as I generally keep the machine on. Phanteks机箱带有三个风扇,但建议我升级进气口,并将机箱的排气风扇升级为BeQuiet BL071 PWM风扇,因为Dual Titans可以散发大量热量。 我注意到我的房间温度比室外温度高出近2-3度,因为我通常会保持机器开机。 To get the best possible airflow, I bought 5 of these. I have put two at the top of the case along with a Phanteks case fan, 2 of them in the front, and one fan at the back of the case. 为了获得最佳的气流,我购买了其中的5个。 我在机箱顶部放置了两个风扇,并在前面放置了一个Phanteks机箱风扇,在机箱前面放置了2个风扇,在机箱背面放置了一个风扇。 This section is not necessary but wanted to put it in for completion. 本部分不是必需的,但希望将其放入完成。 Given all the power we have got, I didn’t want to cheap out on the peripherals. So I got myself an LG 27UK650 4k monitor for content creation, BenQ EX2780Q 1440p 144hz Gaming Monitor for a little bit of gaming, a Mechanical Cherry MX Red Corsair K68 Keyboard and a Corsair M65 Pro Mouse. 鉴于我们拥有的所有功能,我不想廉价购买外围设备。 因此,我得到了一个用于内容创建的LG 27UK650 4k显示器,一个用于游戏的BenQ EX2780Q 1440p 144hz游戏显示器,一个机械Cherry MX Red Corsair K68键盘和一个Corsair M65 Pro鼠标。 And my build is complete. 我的构建完成了。 I will put the price as per the PCPartPicker site as I have gotten my components from different countries and sources. You can also check the part list at the PCPartPicker site: https://pcpartpicker.com/list/zLVjZf 我将按照PCPartPicker站点的价格进行定价,因为我从不同的国家和来源获得了组件。 您也可以在PCPartPicker站点上检查零件列表: https ://pcpartpicker.com/list/zLVjZf As you can see, this is pretty expensive by any means (even after getting the GPUs from NVIDIA), but that is the price you pay for certain afflictions, I guess. 如您所见,无论如何,这都是相当昂贵的(即使是从NVIDIA获得GPU之后),但是我猜这就是您为某些痛苦所付出的代价。 In this post, I talked about all the parts you are going to need to assemble your deep learning rig and my reasons for getting these in particular. 在这篇文章中,我讨论了组装深度学习装备所需的所有部分,以及我特别希望获得这些部分的原因。 You might try to look out for better components or a different design, but this one has been working pretty well for me for quite some time now, and is it fast. 您可能会尝试寻找更好的组件或其他设计,但是这个组件对我来说已经运行了相当长的一段时间了,而且速度很快。 If you want to see how I set up the Deep Learning libraries after setting up the system with these components, you can view this definitive guide for Setting up a Deep Learning Workstation with Ubuntu 18.04 如果要在使用这些组件设置系统后查看我如何设置深度学习库,可以查看此权威指南, 以了解如何使用Ubuntu 18.04 设置深度学习工作站 。 Let me know what you think in the comments. 让我知道您在评论中的想法。 Thanks for the read. I am going to be writing more beginner-friendly posts in the future too. Follow me up at Medium or Subscribe to my blog to be informed about them. As always, I welcome feedback and constructive criticism and can be reached on Twitter @mlwhiz 感谢您的阅读。 我将来也会写更多对初学者友好的文章。 在Medium上关注我,或订阅我的博客以了解有关它们的信息。 与往常一样,我欢迎您提供反馈和建设性的批评,可以在Twitter @mlwhiz上与他们联系。 翻译自: https://towardsdatascience.com/creating-my-first-deep-learning-data-science-workstation-bd39c2f687e2 深度学习数据集制作工作 那么为什么需要工作站呢? (So why the need for a workstation?)
我的构建 (My Build)
1. 英特尔i9 9920x 3.5 GHz 12核处理器 (1. Intel i9 9920x 3.5 GHz 12 core Processor)
2.微星X299 SLI PLUS ATX LGA2066主板 (2. MSI X299 SLI PLUS ATX LGA2066 Motherboard)
3. Noctua NH-D15 chromax.BLACK 82.52 CFM CPU散热器 (3. Noctua NH-D15 chromax.BLACK 82.52 CFM CPU Cooler)
4. Phanteks Enthoo Pro钢化玻璃盒 (4. Phanteks Enthoo Pro Tempered Glass Case)
5.具有3插槽NVLink的双TITAN RTX (5. Dual TITAN RTX with 3 Slot NVLink)
6. Samsung 970 Evo Plus 1 TB NVME固态驱动器 (6. Samsung 970 Evo Plus 1 TB NVME Solid State Drive)
7. Corsair Vengeance LPX 128GB(8x16GB)DDR4 3200 MHz (7. Corsair Vengeance LPX 128GB (8x16GB) DDR4 3200 MHz)
8. 海盗船1200W电源 (8. Corsair 1200W Power Supply)
9.甚至更多的粉丝 (9. Even More Fans)
10.外围设备 (10. Peripherals)
定价???????????? (Pricing ????????????)
最后 (Finally)
最后
以上就是外向鸵鸟为你收集整理的深度学习数据集制作工作_创建我的第一个深度学习+数据科学工作站 那么为什么需要工作站呢? (So why the need for a workstation?) 我的构建 (My Build) 定价???????????? (Pricing ????????????) 最后 (Finally)的全部内容,希望文章能够帮你解决深度学习数据集制作工作_创建我的第一个深度学习+数据科学工作站 那么为什么需要工作站呢? (So why the need for a workstation?) 我的构建 (My Build) 定价???????????? (Pricing ????????????) 最后 (Finally)所遇到的程序开发问题。
如果觉得靠谱客网站的内容还不错,欢迎将靠谱客网站推荐给程序员好友。
发表评论 取消回复