Combining model-based and model-free reinforcement learning approaches, this
paper proposes and analyzes an $\epsilon$-policy gradient algorithm for the
online pricing learning task. The algorithm extends $\epsilon$-greedy algorithm
by replacing greedy exploitation with gradient descent step and facilitates
learning via model inference. We optimize the regret of the proposed algorithm
by quantifying the exploration cost in terms of the exploration probability
$\epsilon$ and the exploitation cost in terms of the gradient descent
optimization and gradient estimation errors. The algorithm achieves an expected
regret of order $\mathcal{O}(\sqrt{T})$ (up to a logarithmic factor) over $T$
trials.