Both $H_2SO_4$ and $HNO_3$ contain hydroxides....and yet these are STRONG acids....
And an acid is typically the hydroxide of a non-metal, whereas a base is typically the hydroxide of a metal.
Non-metals, from the right hand side of as we face it, TEND to polarize the electron towards themselves, thereby weakening the $O-H$ bond of their HYDROXIDES. On the other hand, metals, tend to have their electron density REMOVED by the oxygen centre....
Here we got $B(OH)_3$...and the oxygen TENDS to polarize the electron density of the $O-H$ bond TOWARDS itself...
$(stackrel(delta+)H-stackrel(delta-)O-)_3B$
....and this the $H-O$ is WEAKENED somewhat....$pK_a=9.24$
