Bug with loki-stack chart while adding datasource

[dk03051996]

Hi,
I am creating this issue to let you know that with recent version, I tried adding datasource in separate grafana as http://loki:3100 and try to save and test but it fails with error "Unable to connect with Loki. Please check the server logs for more details." but I am able to see logs in dashboard. This is happening in GKE and kind cluster both as testing environment.
Please take a look as so many can get confused with this.

Getting below error in grafana pods
"logger=tsdb.loki endpoint=checkHealth pluginId=loki dsName=loki dsUID=cdn1r5hikan7kc uname=admin fromAlert=false t=2024-05-30T11:24:42.682252191Z level=info msg="Prepared request to Loki" duration=23.256µs queriesLength=1 stage=prepareRequest runInParallel=false
logger=tsdb.loki endpoint=checkHealth pluginId=loki dsName=loki dsUID=cdn1r5hikan7kc uname=admin fromAlert=false t=2024-05-30T11:24:42.861757897Z level=error msg="Error received from Loki" duration=179.38691ms stage=databaseRequest statusCode=400 contentLength=65 start=1970-01-01T00:00:01Z end=1970-01-01T00:00:04Z step=1s query=vector(1)+vector(1) queryType=instant direction=backward maxLines=0 supportingQueryType=none lokiHost=loki.monitoring.svc.cluster.local:3100 lokiPath=/loki/api/v1/query status=error error="parse error at line 1, col 1: syntax error: unexpected IDENTIFIER" statusSource=downstream
logger=tsdb.loki endpoint=CheckHealth t=2024-05-30T11:24:42.861870966Z level=error msg="Loki health check failed" error="error from loki: parse error at line 1, col 1: syntax error: unexpected IDENTIFIER"
logger=context userId=1 orgId=1 uname=admin t=2024-05-30T11:24:42.861963433Z level=info msg="Request Completed" method=GET path=/api/datasources/uid/cdn1r5hikan7kc/health status=400 remote_addr=127.0.0.1 time_ms=181 duration=181.02212ms size=106 referer=http://localhost:3001/connections/datasources/edit/cdn1r5hikan7kc handler=/api/datasources/uid/:uid/health status_source=server"

[spuyet]

I just got the same issue, the endpoint returns a parser error while we can still send query to the data source.

[dk03051996]

looks like loki-stack is not being maintained properly.

[Sheikh-Abubaker]

loki-stack is deprecated