This is a common issue and could be caused by a number of things. I would first make sure you setup your account correctly - double check all the initial settings (email account, mail server, authentication, etc.). If outgoing mail server authentication is required by your ISP, for example, you'll need to enable this.
The default sending port (SMTP) is port 25. Try changing this to 587. Another option is to change your sending server (SMTP) - for example, to that of your ISP if you were previously using that of your email or hosting provider. You could also, under "advanced settings", increase the server timeout time from 1 to 5 minutes and see if that makes a difference.
You can also try registering a send/receive library file that may have been missing by browsing to Start -> Run -> and typing "regsvr32 inetcomm.dll"
Next thing to check - if you're getting an error that your Outlook profile (.pst) is too large, this is most likely causing the problem. Why? If the file has reached its maximum allowed size, your profile won't receive messages back from the server confirming your message was sent, so it will send again and again. The solution? Free more space by emptying your trash folder (Deleted Items). You may also be able to run maintenance on your profile that will compress your data and free up more space. If it's still too large (over 2GB), you may need to truncate it using the MS PST2GB tool (note that this will result in some data loss).
If that doesn't solve it, the problem might be caused by security software that is checking your email during the sending process. Try temporarily disabling your antivirus, firewall, anti-spyware, etc. - but only one at a time and during the sending process so you can isolate the issue. And of course, be sure to turn them back on when you're done testing!
If none of that works, try contacting your email provider for tech support. If they can't help, you might need to simply create a new Outlook profile.
