Maternal weight loss and breastfeeding

I’m currently exclusively breastfeeding my 4 month old, and I gained about 20lbs during pregnancy. I’m finding it so hard to lose the weight. I walk about three miles a day, I try to stay hydrated, and watch my calories but it’s not shifting. Is this just a breastfeeding thing or a postpartum thing, or something else?