I would strongly recommend against it.
Such a dry climate as southern CA would be really hard. You'd want things all over the house to keep moisture in the air such as humidifiers or decorative fountains.
Going out without a hat would never be possible during the daytime, and long sleeves and pants would be required. You then run the risk of overheating.
I live in Portland, OR where the summers are much like the southern CA climate. I am miserable from late May to early October. I cannot even comprehend what it would be like to live there year round. I've spent a great deal of time in Northern CA and as soon as you go south of Eureka it's no fun at all.
I am female, and was born in 1972 with Lamellar Ichthyosis.