User Panel
User Panel
User Panel
|
Posted: 9/17/2014 12:28:40 PM EDT
[#1]
I'm not quite sure what kind of computations are involved in econometrics, but a good option for just crunching numbers all day is a refurbished server. I bought an 8-core 2.4 GHz Xeon server from this guy: http://stores.ebay.com/MrRackables for $300, that just sits in a closet all day doing matrix math and curve fits. They are designed to run at full CPU load continuously.
The critical thing will be exploiting parallelism in your calculations, if at all possible. If you can speed up what you're doing by a factor of 8, that's potentially a massive time savings. Edit: This one is a potentially nice option- Xeon Server |
|
|
|
Posted: 9/17/2014 6:12:22 PM EDT
[#2]
Can't comment on econometrics, but I do a lot of work on next generation sequencing data and those data sets are large too.
We started off by building a virtual cluster with existing computers in the lab. Nowadays, we've switched over to the Intel 8 Core Haswell chips. It seems to us that having more cores/threads is better than having limited cores at higher frequencies. Especially if your software is geared towards multithreaded applications. High RAM is also the way to go. I agree with the above poster, some of the Xeon server stuff might be right up your alley depending on budget.
|
|
|
|
Posted: 9/17/2014 9:25:32 PM EDT
[#3]
I will be primarily using SAS. The big thing that is slowing me down is simple data manipulations (which take forever with huge data sets). I haven't started trying to do any estimations, but the empirical methods themselves should be pretty simple, and SAS tends to run very fast compared to other software.
|
|
|
|
Posted: 9/18/2014 1:00:38 AM EDT
[#4]
Quote HistoryQuoted:
I will be primarily using SAS. The big thing that is slowing me down is simple data manipulations (which take forever with huge data sets). I haven't started trying to do any estimations, but the empirical methods themselves should be pretty simple, and SAS tends to run very fast compared to other software. Max-ing your system RAM will be imperative with large data sets. The SAS website unfortunately gives no clues as to how/if their code exploits multiple cores/threads. |
|
|
|
Posted: 9/18/2014 10:50:54 AM EDT
[#5]
Quote HistoryQuoted: Max-ing your system RAM will be imperative with large data sets. The SAS website unfortunately gives no clues as to how/if their code exploits multiple cores/threads. Quote HistoryQuoted: Quoted: I will be primarily using SAS. The big thing that is slowing me down is simple data manipulations (which take forever with huge data sets). I haven't started trying to do any estimations, but the empirical methods themselves should be pretty simple, and SAS tends to run very fast compared to other software. Max-ing your system RAM will be imperative with large data sets. The SAS website unfortunately gives no clues as to how/if their code exploits multiple cores/threads. I have no experience at all with rack mounted machines. I just did some googling, and folks said you can run them like a desktop outside of a rack, and that the only big downside is noise (which doesn't bother me too much). ETA: I talked with another grad student in my office who apparently did the same thing. He said he was working for a financial institution when they upgraded their servers. He got one of the old ones for free. He apparently set it up in his basement, and uses it remotely with his laptop. If I went with the option listed above (2.5 ghz processors, 32 gig ram) what programming should I get for it? I have confirmed that I have free access to a linux compatible version of SAS if I wanted to try linux for the first time. ETA2: Another option I'm considering is this. This is basically a case with mother board, an intel xeon processor and slots for tons of memory. Looks like a decent base for a build. I'm thinking I might go with a SSD for software and a HDD for data. I have a spare copy of windows 7 collecting dust.
|
|
|
|
Posted: 9/18/2014 10:14:28 PM EDT
[#6]
Quote HistoryQuoted:
From my reading, it is possible. I don't know how to do it though. I think it can come in handy during the data manipulations (essentially I am modifying and merging a bunch of annual datasets into a time series). I don't know if it will be useful on the estimation itself, but it might. I don't know, since I haven't done my homework on parallel computing. I have no experience at all with rack mounted machines. I just did some googling, and folks said you can run them like a desktop outside of a rack, and that the only big downside is noise (which doesn't bother me too much). ETA: I talked with another grad student in my office who apparently did the same thing. He said he was working for a financial institution when they upgraded their servers. He got one of the old ones for free. He apparently set it up in his basement, and uses it remotely with his laptop. If I went with the option listed above (2.5 ghz processors, 32 gig ram) what programming should I get for it? I have confirmed that I have free access to a linux compatible version of SAS if I wanted to try linux for the first time. ETA2: Another option I'm considering is this. http://www.amazon.com/Lenovo-ThinkServer-70A4001LUX-E3-1225-Processor/dp/B00FE29IWK/ref=sr_1_1?s=electronics&ie=UTF8&qid=1411073858&sr=1-1&keywords=intel+xeon+server This is basically a case with mother board, an intel xeon processor and slots for tons of memory. Looks like a decent base for a build. I'm thinking I might go with a SSD for software and a HDD for data. I have a spare copy of windows 7 collecting dust. Quote HistoryQuoted:
Quoted:
Quoted:
I will be primarily using SAS. The big thing that is slowing me down is simple data manipulations (which take forever with huge data sets). I haven't started trying to do any estimations, but the empirical methods themselves should be pretty simple, and SAS tends to run very fast compared to other software. Max-ing your system RAM will be imperative with large data sets. The SAS website unfortunately gives no clues as to how/if their code exploits multiple cores/threads. I have no experience at all with rack mounted machines. I just did some googling, and folks said you can run them like a desktop outside of a rack, and that the only big downside is noise (which doesn't bother me too much). ETA: I talked with another grad student in my office who apparently did the same thing. He said he was working for a financial institution when they upgraded their servers. He got one of the old ones for free. He apparently set it up in his basement, and uses it remotely with his laptop. If I went with the option listed above (2.5 ghz processors, 32 gig ram) what programming should I get for it? I have confirmed that I have free access to a linux compatible version of SAS if I wanted to try linux for the first time. ETA2: Another option I'm considering is this. http://www.amazon.com/Lenovo-ThinkServer-70A4001LUX-E3-1225-Processor/dp/B00FE29IWK/ref=sr_1_1?s=electronics&ie=UTF8&qid=1411073858&sr=1-1&keywords=intel+xeon+server This is basically a case with mother board, an intel xeon processor and slots for tons of memory. Looks like a decent base for a build. I'm thinking I might go with a SSD for software and a HDD for data. I have a spare copy of windows 7 collecting dust. I run Debian linux on my machine. I don't have it in a rack- didn't see a point. I run the machine totally headless- the only way I talk to it is via ssh over the internet. If you're in the same building, the LAN is probably fast enough that you can display a GUI as well, if that makes it easier. If you're connecting to it from offsite, however, it's probably too slow to use anything but a command line interface. I use git to pull/push data to the server and keep it synced with the data on my laptop (although this isn't really the best use of git). The whole setup works great, and I can have the machine send me an email when a long job is done. As for the machine you linked, you're gonna have a lot of extra expense there. ECC server RAM is not like desktop RAM- loading it up with 32 Gb will be really really expensive. I also don't really see a lot of value in a SSD in a server-type machine. The programs you use should always be kept running anyway, and you're not going to have a lot of high-bandwidth file system access that needs to happen when you're crunching data. It's also only 4 cores, although at a good clock rate. |
|
|
|
Posted: 9/19/2014 3:12:10 PM EDT
[#7]
I'm still trying to determine what slows my computer down so much when running code. I know cooling is a big one, as it overheats often when running long processes. It appears that hard drive utilization was the only characteristic maxing out. Granted this is for data manipulation and not statistics/linear algebra, so I wouldn't expect a lot of processing power going into it.
I'm trying to decide if just getting a big box with good fans is good enough, or if I really do need to load it up with RAM. The main reason I'm thinking of building this like a desktop is that my lap top is my only functioning computer, and I'd like to get a more powerful desktop for daily use (that is capable of running SAS for long periods of time) and using the laptop as a backup (or as a primary when the other computer is running empirical software).
|
|
|
|
Posted: 9/20/2014 11:10:22 PM EDT
[#8]
Quote HistoryQuoted: I'm still trying to determine what slows my computer down so much when running code. I know cooling is a big one, as it overheats often when running long processes. It appears that hard drive utilization was the only characteristic maxing out. Granted this is for data manipulation and not statistics/linear algebra, so I wouldn't expect a lot of processing power going into it. I'm trying to decide if just getting a big box with good fans is good enough, or if I really do need to load it up with RAM. The main reason I'm thinking of building this like a desktop is that my lap top is my only functioning computer, and I'd like to get a more powerful desktop for daily use (that is capable of running SAS for long periods of time) and using the laptop as a backup (or as a primary when the other computer is running empirical software). A case with good airflow is important, but most of the newer chips have very low TDPs, and don't generate that much heat. If CPU overheating is a major concern, just get a Thermaltake or Zalman aftermarket cooler.
|
|
|
Win a FREE Membership!
Sign up for the ARFCOM weekly newsletter and be entered to win a free ARFCOM membership. One new winner* is announced every week!
You will receive an email every Friday morning featuring the latest chatter from the hottest topics, breaking news surrounding legislation, as well as exclusive deals only available to ARFCOM email subscribers.
AR15.COM is the world's largest firearm community and is a gathering place for firearm enthusiasts of all types.
From hunters and military members, to competition shooters and general firearm enthusiasts, we welcome anyone who values and respects the way of the firearm.
Subscribe to our monthly Newsletter to receive firearm news, product discounts from your favorite Industry Partners, and more.
Copyright © 1996-2024 AR15.COM LLC. All Rights Reserved.
Any use of this content without express written consent is prohibited.
AR15.Com reserves the right to overwrite or replace any affiliate, commercial, or monetizable links, posted by users, with our own.
