As long as all public keys in use are based on secp256k1, it doesn't matter. When there will be something else, then Silent Payments could be updated accordingly. Well, for secp256k1, you currently need around 2^128 operations to break a public key. For 160-bit hashes, you need around 2^81 computations, to find a collision, and move coins from identical addresses in at least two different ways, which will undermine trust in all 160-bit addresses, and push people to move to longer hashes, or different output types. Which means, that P2PKH, P2SH, and P2WPKH can be potentially attacked in practice, if someone would build some ASICs, and use some storage for lookup tables. It is doable, but costly. While breaking arbitrary public keys is still more theoretical, than practical, at least for now. In the famous puzzle with weak public keys, 71-bit hashed key is still unsolved. And for public keys, there is 135-bit key. Assuming by the current progress, people are closer to reaching 81-bit hashed key, and having enough power for making 160-bit collisions, than they are to breaking 256-bit random public keys (the puzzle ends on 160-bit public key, but to make things practical, the range of public keys from 161 to 256 bits have to be considered as well; it makes more sense, when keys are not hashed). Of course, quantum progress can change these things, but currently, we have, what we have. And even if someone thinks, that hashing is enough to protect public keys from quantum attacks, then still: 160-bit hashes may be too short for that.
As long as all public keys in use are based on secp256k1, it doesn't matter. When there will be something else, then Silent Payments could be updated accordingly. Well, for secp256k1, you currently need around 2^128 operations to break a public key. For 160-bit hashes, you need around 2^81 computations, to find a collision, and move coins from identical addresses in at least two different ways, which will undermine trust in all 160-bit addresses, and push people to move to longer hashes, or different output types. Which means, that P2PKH, P2SH, and P2WPKH can be potentially attacked in practice, if someone would build some ASICs, and use some storage for lookup tables. It is doable, but costly. While breaking arbitrary public keys is still more theoretical, than practical, at least for now. In the famous puzzle with weak public keys, 71-bit hashed key is still unsolved. And for public keys, there is 135-bit key. Assuming by the current progress, people are closer to reaching 81-bit hashed key, and having enough power for making 160-bit collisions, than they are to breaking 256-bit random public keys (the puzzle ends on 160-bit public key, but to make things practical, the range of public keys from 161 to 256 bits have to be considered as well; it makes more sense, when keys are not hashed). Of course, quantum progress can change these things, but currently, we have, what we have. And even if someone thinks, that hashing is enough to protect public keys from quantum attacks, then still: 160-bit hashes may be too short for that.

As long as all public keys in use are based on secp256k1, it doesn't matter. When there will be something else, then Silent Payments could be updated accordingly. Well, for secp256k1, you currently need around 2^128 operations to break a public key. For 160-bit hashes, you need around 2^81 computations, to find a collision, and move coins from identical addresses in at least two different ways, which will undermine trust in all 160-bit addresses, and push people to move to longer hashes, or different output types. Which means, that P2PKH, P2SH, and P2WPKH can be potentially attacked in practice, if someone would build some ASICs, and use some storage for lookup tables. It is doable, but costly. While breaking arbitrary public keys is still more theoretical, than practical, at least for now. In the famous puzzle with weak public keys, 71-bit hashed key is still unsolved. And for public keys, there is 135-bit key. Assuming by the current progress, people are closer to reaching 81-bit hashed key, and having enough power for making 160-bit collisions, than they are to breaking 256-bit random public keys (the puzzle ends on 160-bit public key, but to make things practical, the range of public keys from 161 to 256 bits have to be considered as well; it makes more sense, when keys are not hashed). Of course, quantum progress can change these things, but currently, we have, what we have. And even if someone thinks, that hashing is enough to protect public keys from quantum attacks, then still: 160-bit hashes may be too short for that.

As long as all public keys in use are based on secp256k1, it doesn't matter. When there will be something else, then Silent Payments could be updated accordingly. Well, for secp256k1, you currently need around 2^128 operations to break a public key. For 160-bit hashes, you need around 2^81 computations, to find a collision, and move coins from identical addresses in at least two different ways, which will undermine trust in all 160-bit addresses, and push people to move to longer hashes, or different output types. Which means, that P2PKH, P2SH, and P2WPKH can be potentially attacked in practice, if someone would build some ASICs, and use some storage for lookup tables. It is doable, but costly. While breaking arbitrary public keys is still more theoretical, than practical, at least for now. In the famous puzzle with weak public keys, 71-bit hashed key is still unsolved. And for public keys, there is 135-bit key. Assuming by the current progress, people are closer to reaching 81-bit hashed key, and having enough power for making 160-bit collisions, than they are to breaking 256-bit random public keys (the puzzle ends on 160-bit public key, but to make things practical, the range of public keys from 161 to 256 bits have to be considered as well; it makes more sense, when keys are not hashed). Of course, quantum progress can change these things, but currently, we have, what we have. And even if someone thinks, that hashing is enough to protect public keys from quantum attacks, then still: 160-bit hashes may be too short for that.