Couple of reasons, but to address Harry Potter in particular, those films were the result of several directors and began before all the books were written so things were ...inconsistent to say the least, some points were skipped and left later directors on a bind. One example is that in the books the first time Harry kills someone is a big deal for him emotionally, but this would make no sense in the film since it had him literally kill someone in the first movie.

More generally, some things don’t translate. Good luck literally adapting what is just internal monologs, or keeping pacing the same as a book that relied on long passages of detailed descriptions, or any story that won’t fit in a ~100min 3 act structure.