Dear Colleagues,
We are currently in the process of building a cluster, which will be mainly used for VASP simulations. Those include both DFT and hybrid-DFT, GW and NEB simulations.
I would like to ask your advice regarding the appropriate hardware:
The head node: two eight-core Intel Xeon E5-2650 processors, 64 GB memory, and two 300GB SAS disks.
Computing nodes: eight-core Intel Xeon E5-2650, 64GB memory, 2TB SATA disk.
What would be appropriate for the scratch, SDD, HDD drive?
I would appreciate any comments and suggestions.
Thank you in advance,
Kind regards,
Monica
Building cluster to run VASP
Moderators: Global Moderator, Moderator
-
mkosa
- Newbie
- Posts: 5
- Joined: Sun Apr 22, 2012 8:45 am
- Contact:
Building cluster to run VASP
Last edited by mkosa on Wed Apr 25, 2012 10:59 am, edited 1 time in total.
-
alex
- Hero Member
- Posts: 583
- Joined: Tue Nov 16, 2004 2:21 pm
- License Nr.: 5-67
- Location: Germany
Building cluster to run VASP
Hello Monica,
some experiences:
Memory:
- for plain DFT ~1 GB/core is sufficient. If your calculation gets larger, use more cores. Scaling is still good.
- Hybrid DFT is more demanding. If I do such calculations (sometimes only), I go for, e.g. only half the cores per box using full memory
- GW: no idea about that.
CPUs:
You might wish to consider AMD Opteron also. Check SpecFP rates to get a proper comparison with Intel. Better: Ask your local dealer for two test sets. It might save quite a bit of money! I'm very happy with a 8C 2.2 GHz Opteron. In my experience 12C was worse than 8C, because they have the same number of memory channels/interfaces. So the performance dropped dramatically (30-50%) with even only medium sized (~100 atoms) systems. One thing to check!
Scratch:
In my experience (DFT and hybrid DFT) this is no issue. I'm going along with some cheap HDD. VASP does not write large files on a regular basis (except you ask for it), because the performance would break down. Again: No idea about GW though ...
Network:
Get anything which is faster (better: lower latency) than GBit! This is really an issue. Infiniband or Myrinet is good. I never tried 10GBit Ethernet (the cost of the switch was astronomical at the time of my last purchase).
Cheers,
alex
PS: I've got a bunch of 2x AMD Opteron 8C 2.2 GHz, 32 GB memory, 2 HDD (160 GB) for RAID1 and Infiniband (cheapest card available). The machine is called HP BL465c G7. Server is similar, but more HDD and memory, but no IB network.
some experiences:
Memory:
- for plain DFT ~1 GB/core is sufficient. If your calculation gets larger, use more cores. Scaling is still good.
- Hybrid DFT is more demanding. If I do such calculations (sometimes only), I go for, e.g. only half the cores per box using full memory
- GW: no idea about that.
CPUs:
You might wish to consider AMD Opteron also. Check SpecFP rates to get a proper comparison with Intel. Better: Ask your local dealer for two test sets. It might save quite a bit of money! I'm very happy with a 8C 2.2 GHz Opteron. In my experience 12C was worse than 8C, because they have the same number of memory channels/interfaces. So the performance dropped dramatically (30-50%) with even only medium sized (~100 atoms) systems. One thing to check!
Scratch:
In my experience (DFT and hybrid DFT) this is no issue. I'm going along with some cheap HDD. VASP does not write large files on a regular basis (except you ask for it), because the performance would break down. Again: No idea about GW though ...
Network:
Get anything which is faster (better: lower latency) than GBit! This is really an issue. Infiniband or Myrinet is good. I never tried 10GBit Ethernet (the cost of the switch was astronomical at the time of my last purchase).
Cheers,
alex
PS: I've got a bunch of 2x AMD Opteron 8C 2.2 GHz, 32 GB memory, 2 HDD (160 GB) for RAID1 and Infiniband (cheapest card available). The machine is called HP BL465c G7. Server is similar, but more HDD and memory, but no IB network.
Last edited by alex on Thu Apr 26, 2012 9:51 am, edited 1 time in total.
-
mkosa
- Newbie
- Posts: 5
- Joined: Sun Apr 22, 2012 8:45 am
- Contact:
Building cluster to run VASP
Hi Alex,
Thank you very much!
Best,
Monica
Thank you very much!
Best,
Monica
Last edited by mkosa on Mon Apr 30, 2012 7:39 am, edited 1 time in total.