I might say something like "I am sorry for your loss. While your relationship with him was difficult, the loss of a parent has impact and I hope you find healing during this time." Or perhaps say "...while he was difficult, the loss of a parent has impact..."

His death may have little impact on their lives but its bound to have some effect. And in reality, it may have a BIG impact on their lives. People who have a difficult relationship with a dead parent oftentimes struggle the most with that death because they are mourning all over again the parent that they never had.

The trick here is that you don't really know how they feel about this so it's harder to respond than to a conventional death. I hope no one would take my words as an assumption, but that's why this is difficult to write.