Have you heard of the openUPS?

http://www.mini-box.com/OpenUPS

That is awesome. But expensive. Didn't knew about. Thanks for the tip anyway ! :)

Ah, sorry, i assumed you were where i am. First, how much load are we talking here, <200mA or 2A? this will give use varying options. i.e. a step up/step down voltage regulator:
https://www.pololu.com/product/2096, this is 200mA,  or this one:
https://www.servocity.com/html/12v_step_up___down_voltage_reg.html#.VVZXqPlVhBc this one does 2A. 

these have the added benefit of evening out your voltage over time, but there are some losses too. they seem to max out at about 90% efficiency, but a ~12v source should always be above 80% 

It is used to power an audio amplifier that is rated 12V DC, 5 Amps but I may not use at its maximum output. I may throw a Raspberry Pi in this so let's say big maximum between 5 and 6 amps. Thank you for the link. I will investigate the components on the circuit to see if it is able to get a wider range than strictly 12V. Maybe I can avoid using a regulator if it can takes 14V at its maximum voltage.

Here you go, 6A of power for you

http://www.ebay.com/itm/2-in-1-Step-up-Step-down-100W-6A-5-30V-to-1-24V-12V-19v-DC-Converter-Regulator-/321340548236?rmvSB=true

you just have to set it to 12v on the output via the small pot on output side(its underneath, they show it in the second pic on the ebay listing)

in my limited experience, if you apply a charge current to a lead acid, you can continue to pull off of it, especially if it is a small load. just make sure whatever amp charger you get exceeds your load, and you should be fine, with the understanding that only the supplied amps minus your load will actually be charging your battery.

Good luck!

My concern is that if the charger is supplying 14.4V (for example) to charge the battery, then my circuit will also get 14.4V and I'm not sure it can stand this voltage. I was more thinking about a charger with a 12V output that will switch whether the mains supply is on or off. When mains is on, it will give 12V DC from its own power supply, when mains is off, it will give the voltage from the battery. Mains in my case is 230V AC.

A 12 Volt battery is 'nominally' 12 Volts. It will drop lower than 12 Volts but only when getting very flat. So it depends on how 'stable' you need the output to be. For it to be specificity 12Volts then you would need a boost regulator which is complicated. 

My 'observations' of car batteries -

11.8 Volts - no load - no charge - probably not quite enough to start the car but might be

12.2 Volts to 12.4 Volts - no load - no charge - normal charged condition

13.8 Volts - charging

You will find power supplies made for car radios and transceivers have an output of 13.8 Volts as that is what you should find when a car battery is charging. 

A 12 Volt battery charger shouldn't get over 13.8 Volts but a cheap one may. 

So it all depends on the load. How many Amps? and voltage sensitivity. 

You can't regulate 12 Volts from 12 Volts with a normal regulator because you need some overhead voltage for the regulator to work. You could use a Boost regulator but they normally go from 6 Volts to 12 Volts or 12 Volts to 24 Volts but not 12 Volts to 12 Volts

Thank you for your details. As explained above, I may need between 5 and 6 amps and will investigate the components to see if it can take more than 12V. It's chinese crap so it's lacking some informations about the sensitivity and the tolerance. I have to figure it out by myself.