The WordPress logo
WordPress is a widely used blogging platform and, more recently, a CMS. It's written in PHP and it's widely customizable with thousands of plugins and themes.
First of all, you need to download the latest version of WordPress from the official site. When that is done you have to extract the archive and upload the files using FTP. If you want to have your blog in the root of the domain, upload all files to public_html. If you rather have it in a folder, subdomain or an addon domain, rename the folder of the extracted archive to something of your choice, like blog, and upload that folder to your public_html folder (make sure there isn't a folder in the folder, the php files etc should be visible when you go inside the "blog" folder). Point subdomains or addon domains to this folder.
Now when all the files has been uploaded, navigate to the files in your browser. You will be redirected to the WordPress installation. Follow trough the steps, fill in your MySQL database information and other details it asks for. When this is done, you have your own blog!
Plugins and themes
There are lot of plugins, addons and themes for WordPress located on the WordPress site. When installing a plugin, you need to upload the files to the wp-content/plugins directory. Themes goes into the wp-content/themes directory.