If your IdP lets you download the metadata XML file directly but does not provide a URL, you will need to publish that file somewhere public (it should not contain sensitive information). For example, you can upload to a public S3 bucket and then give Panther the S3 URL.