I found this topic a little confusing at first, because the explanations given weren’t clear enough. So I’ll try to explain it simply, at least at the level that I understand it.
These are just applications. They can be web or desktop, it doesn’t really matter. These applications are what a user will attempt to access. So, for example, a user will browse to mail.google.com to access their Gmail account.
Identity Providers are services designed to manage identities, and perform authorisations to access resources. So continuing my Gmail example, once the user reaches mail.google.com Gmail will redirect them to accounts.google.com to get the user to log in. The user will provide their username+password, and then they’ll be redirected back to mail.google.com and authorised to view their email.
The frustrating part of the explanations is that they often talk about how IdPs and SPs are almost the same thing. That they can overlap. This is definitely true.
For example, an SP can also be an IdP. If an application isn’t connected to an external IdP, it’ll usually have some sort of internal identity data store to manage people logging in and interacting with the application. This is the traditional way that applications are made on the web.
And in the inverse, IdPs can be SPs. A good example is Facebook. You create an account with Facebook to access the site (the Service Provider), but you can also use that account as an Identity for other Service Providers.
I hope this clears up the concepts a little better.