Illinois security deposit problem?

I'm an Illinois resident leaving a rental house. When I rented the house, it was still full of the previous renters belongings. The house was also needing to be cleaned and painted. The landlord moved the stuff from the house into the garage, and we made a verbal agreement that if we only paid half of the deposit then we would help with the cleaning and painting. We paid our first months rent and half of the deposit, then we purchased paint and rollers and drop cloths for the house. My wife and I were both working 2 jobs at the time and didnt have much time for working on the house, but we did go in the day after we made our payment and painted the master bedroom. We both had to work the next day and we returned the day after to work some more. We found that the landlord had gone in and finnished painting the downstairs, leaving 2 rooms upstairs to do. However, the landlord took home our drop cloths and rollers so we could not work on those rooms that day. The upstairs is a converted attic and has no heat or air and was pretty uncomfortable year round. Then between work and my wife getting pregnant and giving birth to our first child, we had no time or money to paint the upstairs. We gave the landlord our 30 notice last month, we had no lease, and we are now moved out. In the year we lived there the garage was never cleaned out, even though we had the understanding that the stuff would be moved into a storage unit so that we would have access to and use of the garage. The landlord also lit a boiler that the electric company deemed a fire hazard and refused to light, and he never fixed the horible plumbing that we reported as bad when we moved in. Now the landlord wants 400 dollars for the deposit because the upstairs wasnt painted. I reminded him that I had paid him 200 when we moved in, so now he wants 200 dollars to finnish the painting. I told him that that is not what a deposit is for, and he countered that I agreed to paint the house for the withholding of the deposit. He also told me three times that if I dont give him the 200 dollars that he was "going to come after me". I left the house with no damage and in the same condition as when I moved in, except I cleaned it before leaving. Can he do this? I'm under the impression that a deposit can only be held for damages that I made. As far as I'm aware, the landlord should owe me 200 dollars.