there may not even be any real solution. Well if her problems are mostly due to drunkeness the solution is sobriety but no easy way to make that happen, but dementia is another thing. See family dysfunction can have solutions, but actual dementia can't really be shoved into that category, if someone isn't acting right because their brain is going it's not a psychological problem.
They probably will get the house if it's in the will (if she doesn't burn it down first! install many smoke detectors is the best I can think of). The house going to Medicaid law has been changed. How do I know? I've discussed the issue with an estate attorney when my mom's estate has been discussed with me (my mom does not have dementia, but it does run in the family).
However on the Medicaid paying for a nursing home issue, it is unlikely to risk the home, however I don't know how easy it really is to get and how many time limits and other restrictions you are going to hit, and so your choices here are really going to be limited due to limited means. Even if they weren't nursing homes aren't good places (they are where people go to die seems accurate enough to me). If she has dementia killing herself slowly with cigs may not be that bad an option. There is the hiring a caretaker to come in at least for a few hours a week/day/etc. but who has means to pay for that here, clearly not SIL and BIL it seems to me, but do the other family members? They do have to agree on a plan, but money is going to set limits, the finances of caring for old demented people is also not just a family dysfunction problem but a real issue. Yes of course investigate what you can get from Medicare/Medicaid but I think it is likely you are going to find it's not that generous.