dmitri.shuralyov.com/text/kebabcase

Dep ensure fails dmitri.shuralyov.com/text/kebabcase#3

Closedxdimedrolx opened this issue 5 years ago
xdimedrolx commented 5 years ago

I'm getting this error:

$ dep ensure -v  -add dmitri.shuralyov.com/text/kebabcase
Fetching sources...
(1/1) dmitri.shuralyov.com/text/kebabcase

Failed to add the dependencies:

  ✗ could not infer project root from dependency path: dmitri.shuralyov.com/text/kebabcase: unable to deduce repository and source type for "dmitri.shuralyov.com/text/kebabcase": unable to read metadata: multiple meta tags match import path "dmitri.shuralyov.com/text/kebabcase"

adding dependencies failed

It used to work a couple of days ago.

Write Preview Markdown
dmitshur commented 5 years ago · edited

Thanks for the report, I'll take a look.

The error message mentions:

multiple meta tags match import path "dmitri.shuralyov.com/text/kebabcase"

This sounds like dep isn't able to parse the "mod" VCS type correctly.

Yep, confirmed that's the root cause. Filed issue golang/dep#2151 for dep.

Write Preview Markdown
msenmurugan commented 5 years ago

dmitshur, whether there is any workaround for this issue? Since gobuffalo used this repo, we are facing the same issue in our code.

Write Preview Markdown
dmitshur commented 5 years ago · edited

msenmurugan, I'm not seeing where gobuffalo is using this Go package. Which version of gobuffalo is it? Can you please point me to it? Is there an open issue there about it?

Same question about your code, can you point me to where you're facing the problem, or is it closed source? Is the dmitri.shuralyov.com/text/kebabcase package used directly or indirectly?

It's hard for me to suggest workarounds for your specific use case because I don't have enough information about it.

Some options may include updating dependencies in case a newer version no longer requires this package, trying a different dependency management tool (cmd/go and its support for modules would be my recommendation), or waiting for your current dependency management tool to resolve this problem.

As far as I know, this package is following the protocol described in https://golang.org/cmd/go/#hdr-Remote_import_paths correctly, and because of the heavy load on my server, it's hard for me to go back to not using the "mod" go-import meta tag. It's something I can consider, but I need more information about where the problem is happening to make that call.

Write Preview Markdown
dmitshur commented 5 years ago

I'm not seeing where gobuffalo is using this Go package. Which version of gobuffalo is it?

I see now it was used earlier on, but has been removed on Oct 9, 2018 in PR gobuffalo/buffalo#1364. Perhaps another workaround is to update to that version.

Write Preview Markdown
ddoan commented 5 years ago

It's used by https://github.com/gobuffalo/github_flavored_markdown (in internal/shurcooL/octicon/generate.go). github_flavored_markdown is used all over Buffalo, even in the latest alpha release.

$ cd vendor

$ grep -r github_flavored_markdown * | awk -F: '/\.go/ {print $1}' | sort -u
github.com/gobuffalo/buffalo/buffalo/cmd/fix/fix.go
github.com/gobuffalo/buffalo/render/html.go
github.com/gobuffalo/plush/markdown_helper.go
github.com/gobuffalo/plush/partial_helper.go
Write Preview Markdown
dmitshur commented 5 years ago

I see, thanks for pointing that out ddoan. That means that plan to update dependencies wasn't going to help.

I've sent PR golang/dep#2152 to dep that resolves this issue.

I've tested it, and it allows dep ensure to work successfully on this package.

Hopefully it gets reviewed and merged soon, but if you'd like, you can compile dep from source with that PR checked out and start using it sooner.

Write Preview Markdown
dmitshur commented 5 years ago · edited

The dep issue has been fixed (as can be seen here), so I'll close this.

Thanks again for the report. Please let me know if something else comes up.

Write Preview Markdown
dmitshur closed this 5 years ago
ribice commented 5 years ago

@dmitshur: For some reason using the provided Dockerfile by GoBuffalo I'm still having the same issue.

Locally the issue was resolved by updating to latest dep release.

What's strange is that the Dockerfile (I tried 0.14.3, and development) fetches the binary by go get -u github.com/golang/dep/cmd/dep.

It fails here:

remote: Step 9/20 : RUN buffalo build --static -o /bin/app
remote:  ---> Running in 0e312ab20dcb
remote: grouped write of manifest, lock and vendor: error while writing out vendor tree: failed to write dep tree: failed to export dmitri.shuralyov.com/text/kebabcase: unable to deduce repository and source type for "dmitri.shuralyov.com/text/kebabcase": unable to read metadata: multiple meta tags match import path "dmitri.shuralyov.com/text/kebabcase"
Write Preview Markdown
dmitshur commented 5 years ago

@ribice I've looked in the GoBuffalo project, and I see it calls out to dep in a few places, but I don't see an old version specified anywhere. Maybe it's a layer caching issue (i.e., perhaps that go get -u step didn't actually re-run)?

Can you share the command you ran that I can reproduce this with?

Write Preview Markdown
dmitshur commented 5 years ago · edited

@ribice Oh, I just noticed the latest dep release tag is missing the "v" prefix! See https://github.com/golang/dep/releases. That means doing go get -u in module mode will actually install v0.5.1, without the fix, since it ignores 0.5.2 tag due to it not being a valid semver. That might be what's causing this.

I'll ask Kevin to create a new tag (of the same commit) with the "v" prefix.

Write Preview Markdown
ribice commented 5 years ago · edited

@dmitshur: Thanks. Will keep you updated once they patch the version with prefix.

Write Preview Markdown
dmitshur commented 5 years ago

@ribice There's a new dep release tag v0.5.3 published now. Can you please try again and let me know if the issue is resolved now?

Write Preview Markdown
ribice commented 5 years ago

I just did and unfortunately the issue is still there.

remote: Step 9/20 : RUN buffalo build --static -o /bin/app
remote:  ---> Running in 5327276ef857
remote: grouped write of manifest, lock and vendor: error while writing out vendor tree: failed to write dep tree: failed to export dmitri.shuralyov.com/text/kebabcase: unable to deduce repository and source type for "dmitri.shuralyov.com/text/kebabcase": unable to read metadata: multiple meta tags match import path "dmitri.shuralyov.com/text/kebabcase"
Write Preview Markdown
dmitshur reopened this 5 years ago
dmitshur commented 5 years ago

It must be something else then. I’ll reopen this.

How can I reproduce it? What was the command you ran?

Write Preview Markdown
ribice commented 5 years ago

I use heroku for deploying my buffalo app, so it's just git push heroku. It deploys as a container (not using their Go runtime).

The Dockerfile was generated by buffalo-heroku.

It looks like:

FROM gobuffalo/buffalo:v0.14.3 as builder

RUN mkdir -p $GOPATH/src/github.com/ribice/web
WORKDIR $GOPATH/src/github.com/ribice/web

# this will cache the npm install step, unless package.json changes
ADD package.json .
ADD yarn.lock .
RUN yarn install --no-progress
ADD . .
RUN go get $(go list ./... | grep -v /vendor/)
RUN buffalo build --static -o /bin/app

FROM alpine
RUN apk add --no-cache curl
RUN apk add --no-cache curl
RUN apk add --no-cache curl
RUN apk add --no-cache bash
RUN apk add --no-cache ca-certificates

WORKDIR /bin/

COPY --from=builder /bin/app .

# Uncomment to run the binary in "production" mode:
# ENV GO_ENV=production

# Bind the app to 0.0.0.0 so it can be seen from outside the container
ENV ADDR=0.0.0.0

EXPOSE 3000

# Uncomment to run the migrations before running the binary:
# CMD /bin/app migrate; /bin/app
CMD exec /bin/app

It fails on RUN buffalo build --static -o /bin/app.

Funnily enough, I tried building the Docker image locally using the provided Dockerfile and it executed without problem. Could somehow the dependencies that are being fetched by gobuffalo/buffalo be cached, so that it pulls an older version of dep?

Write Preview Markdown
dmitshur commented 5 years ago

I'm not an expert on Docker so I could be mistaken about some of this, but my understanding is that it does cache pretty aggressively. I suspect there's a good chance that's what's happening here.

As far as I know, it may not re-run a RUN command unless its contents actually change. Perhaps you can try changing that RUN line slightly to trigger that. Another thing to try is adding a RUN just before that prints the dep version, like:

RUN dep version
RUN buffalo build --static -o /bin/app

And make sure it's v0.5.3 rather than something older.

Docker has a --no-cache flag, but since you're working with Heroku, you may need to accomplish that differently. I'm finding this resource: https://help.heroku.com/18PI5RSY/how-do-i-clear-the-build-cache.

Please let me know if that helps.

Write Preview Markdown
ribice commented 5 years ago · edited

Dep version was printing devel which served no purpose.

I added a step befpre buffalo build:

RUN go get -u github.com/golang/dep/cmd/dep

And now it works fine. Thanks for your support!

Write Preview Markdown
dmitshur commented 5 years ago

It sounds like caching was indeed the cause. Sorry for the trouble.

Glad to hear it's resolved now. :) I'll close this.

Write Preview Markdown
dmitshur closed this 5 years ago
to comment.