http://onlinelibrary.wiley.com/doi/10.1890/070151/full
http://www.sciencedirect.com/science/article/pii/S0921800904...
http://online.liebertpub.com/doi/full/10.1089/vbz.2011.0814
The second one of those estimates the economic damage caused by invasive species in the US as $97 billion during the 20th century.
If you would rather like something written for the general public, here is an info page from the Australian government - or you could just try taking your imported seeds with you in your hand luggage when flying into Sydney, and see how well that goes.
http://www.environment.gov.au/biodiversity/invasive-species/...
The seeds you buy at your local garden center are subject to national and international laws, regulations and agreements specifically written based on work of hundreds of scientists to prevent as much as possible the introduction of diseases and other bad things that don't exist where you live. The garden center has to follow these regulations (and document compliance) or they get fined. When you buy seeds on eBay or AliExpress or wherever, there simply are no regulations on the sellers. Do you dispute this simple fact?
Please tell which fungi, bacteria and insects spread through the vector you propose. Where do you think the seed you buy in your local garden center comes from?