You can clean it with a pressure washer and household siding cleaner, or just use a car washing tool. There are a bunch of inexpensive tools that attach to a hose and have a brush and often can mix soap or cleaner.
The paint will continue to chalk, and that is a normal self-cleaning mechanism. Remember you can paint aluminum siding very easily. Best results are airless spray applied latex, but hand-painting will work. Just be careful about setting ladders against the siding.