Not necessarily. Take jQuery: many sites use it. If both sites A and B load it from a CDN, and I've already visited site A, then when I visit site B, jQuery is already in my cache: we might not even need to request it, despite never having been to site B before.

However, traditionally, the CDN now controls the content of your JS, and could inject whatever they want into it. That's where this proposal comes in…

(Of course, if it isn't in the cache, then you might need as much as a DNS lookup+TCP connect+a TLS handshake to another host… tradeoffs. HTTP/1.x is also limited to n connections to a DNS name at time, so you can parallelize requests by hosting across multiple domains, such as a CDN, but I find this argument less compelling.)

Different sites might use different versions of a library. Also the cache gets polluted very quickly because many sites add caching headers for every request and cache size is limited.

If average page size is 500 Kb and jquery size is 30 Kb gzipped you do not save much by hosting is on a CDN. What you get is more DNS requests, more downtime when that server fails or stalls and give out data about your users.

I think it is easier just to host everything on your own server.

Exactly, people are giving up fundamental security for very little free resources and a tiny, tiny one-off performance improvement. But, what people also don't seem to care about is that these CDN's are actually created to track both users and site traffic as well. No bank uses (or should be using) third party CDN's, but other sites seem to not care.