If you are doing your programming on Mac or Linux, you should already have everything you need already installed. On Mac, under your settings, make sure your web server (Apache) and database (MySQL) are set to automatically start when you turn on your computer. Most Linux distributions have this already done by default; but if you aren't sure, you can check in your package manager, or with members of your distro's forum, to find out.
Windows does not come with a proper web server built-in, so you will have to download one from the internet. XAMPP is a collection of tools originally designed for Windows, but now has packages for Mac and Linux as well. Start by downloading the best package for your operating system and security or skill level from http://xampp.org or http://www.apachefriends.org. When you install the package, make sure the location chosen has permission to regularly make changes to the files in that directory. If you are the only user of the computer you are installing XAMPP on, it will probably be most convenient to use your own (My )Documents folder. If your installation didn't already, make two shortcuts on your desktop: one for the XAMPP Control Panel pointing to Documents\xampp\xampp-control.exe, and the other for the xampp\htdocs folder. XAMPP can be set up to run as a service on Windows the same way it is for Mac or Linux, so you never have to worry about whether it is running or not. There are, however, many reason why you may not want to do that. Generally, when you are ready to test your development, open the XAMPP Control Panel and click "Start" next to Apache and MySQL. When you are ready to shut off your computer, first go back to the same place and click "Stop" next to both services and wait until they show they are shut down. To get out of the XAMPP Control Panel completely, you have to use the "Exit" button, and not the red "X" in the corner.
In addition to the web server software, you will need an editor to write your programs and scripts in. While it's possible to use a simple TextEdit or Notepad application, there are a number of features that make the job of programming a lot easier. Some are free, and some are fairly expensive; but so far I haven't found any that work on every platform that has all the features I would love in a programmer's editor. There are a few I recommend, however. For Mac, you can use XCode. Though it is considered part of the OS X system, it still has to be downloaded for free from the App store and installed. If using KDE on Linux, you should already have Kate installed. For Windows, you will again have to resort to the internet. Notepad++ has many of the features required in a fairly simple editor. There are other, much more complicated editors and Integrated Development Environments (IDEs) out there, including Zend Studio, Eclipse, EMACS, and others. But these have so many features and commands to learn, it makes it hard to be productive out of the box.
There are other tools that we will discuss in each of the courses that are indispensable for any developer of merit. You will need an FTP client, and possibly server. You will also need a testing framework, and a version control repository with the client software to match. Don't worry if you don't know what these are just yet, as they only become important after you've really gotten well into your projects.