You need only concern yourself with the multiplier.
The i5-2500k bus/core ratio is 33 I believe, which means the bus would be 100Mhz. Head into the BIOS/UEFI and change the multiplier to 40. There's your 4Ghz overclock. Not too tough, eh?
Here's the real question though. Are you fine with speed-step and power saving features, or do you want the CPU @ 4Ghz all the time?
Pretty much this. You can safely bump if up to 4.4Ghz (multiplier of 44) provided you have adequate cooling without too much issue. The voltage is automatically determined among other things. From here you may need to change other parameters.
For the downclocking, default settings Windows will take care of this. The power saving issues aren't much of an issue since this is a relativity low OC.