Themes usually come with one or more menu locations and you can also add menus in the right/left hand column as a widget. Many themes have a Top Menu (probably goes across the top of the page either above or below the top banner) and a Social Links Menu (probably goes bottom right and meant for links to twitter etc)
Your top menu usually leads to the main pages of your website.
The side menu more often lists categories, latest posts, etc.
To Add Menus to Your Site
Dashboard > Appearance > Menus
Create one called “Top Menu” you can always change it later.
Add the pages you want included. This is your main static pages. (You’ll also see options for adding categories, custom links and posts.)
With your posts, adding them all to a main menu would quickly become unmanageable. This is where the different way wordpress treats pages and posts comes in. Posts are automatically grouped into categories by WordPress. A post can be in more than one category, and you can change the categories without changing the body of the post. Create one or more categories.
To Add Categories to Your Site
Dashboard > Posts > Categories.
You could just start with one category (blog / articles) and later expand into multiple categories or subcategories.
When you’ve created a category, you then have the option to include it in your menus or a list of categories in the sidebar.
But people often create a widget for their blog categories and other widgets to go in the right sidebar. This might include a category listing, recent posts, that calendar widget, tag clouds etc.
Dashboard > Appearance > Widgets lists them all
If you use Custom Menu widget it will be exactly as you say.
If you use a Categories widget it will list all your categories IF they have posts in them.
Categories v Tags. Think of categories like the contents of a book and Tags like an index.
I tend to use the dashboard out of habit but you can also do a lot of the above with the newer “Customise” option.