So, you can upgrade most of the hardware parts of your PC! Wait - there are a few constraints. Let's talk more before you jump out and buy a new CPU.
Some of things you will normally want to upgrade, especially if you have an older 486 or an early Pentium, and to a degree even more recent PCs are: RAM, BIOS, Hard Drives, CD-ROM, and probably a Removable High Capacity drive such as a JAZ (1 GB) or SyJet (1.5GB) drive.
First, some considerations: You cannot change a 486SX into a 486DX4, or a Pentium P75 into a Pentium II. You will have to observe some specifics or just buy a new system. You cannot change from a 30-pin SIMM to a 72-pin SIMM memory configuration without a new motherboard. You cannot change from FPM RAM to SDRAM or EDO RAM without a BIOS and SIMM/DIMM slots that accepts them, or a new motherboard. You cannot change from from an ISA bus to a PCI bus without purchasing a new motherboard or a new PC! Sorry. You also are limited in many cases as to what you can do because of only a few expansion slots on your motherboard.
Some of the areas in your PC that you can upgrade are: